{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Large-Scale Stochastic Variational GP Regression (CUDA)\n",
    "\n",
    "## Overview\n",
    "\n",
    "In this notebook, we'll give an overview of how to use SVGP stochastic variational regression ((https://arxiv.org/pdf/1411.2005.pdf)) to rapidly train using minibatches on the `3droad` UCI dataset with hundreds of thousands of training examples. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import torch\n",
    "import gpytorch\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "# Make plots inline\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading Data\n",
    "\n",
    "For this example notebook, we'll be using the `song` UCI dataset used in the paper. Running the next cell downloads a copy of the dataset that has already been scaled and normalized appropriately. For this notebook, we'll simply be splitting the data using the first 80% of the data as training and the last 20% as testing.\n",
    "\n",
    "**Note**: Running the next cell will attempt to download a **~136 MB** file to the current directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "import os.path\n",
    "from scipy.io import loadmat\n",
    "from math import floor\n",
    "\n",
    "if not os.path.isfile('3droad.mat'):\n",
    "    print('Downloading \\'3droad\\' UCI dataset...')\n",
    "    urllib.request.urlretrieve('https://www.dropbox.com/s/f6ow1i59oqx05pl/3droad.mat?dl=1', '3droad.mat')\n",
    "    \n",
    "data = torch.Tensor(loadmat('3droad.mat')['data'])\n",
    "X = data[:, :-1]\n",
    "X = X - X.min(0)[0]\n",
    "X = 2 * (X / X.max(0)[0]) - 1\n",
    "y = data[:, -1]\n",
    "\n",
    "# Use the first 80% of the data for training, and the last 20% for testing.\n",
    "train_n = int(floor(0.8*len(X)))\n",
    "\n",
    "train_x = X[:train_n, :].contiguous().cuda()\n",
    "train_y = y[:train_n].contiguous().cuda()\n",
    "\n",
    "test_x = X[train_n:, :].contiguous().cuda()\n",
    "test_y = y[train_n:].contiguous().cuda()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a DataLoader\n",
    "\n",
    "The next step is to create a torch `DataLoader` that will handle getting us random minibatches of data. This involves using the standard `TensorDataset` and `DataLoader` modules provided by PyTorch.\n",
    "\n",
    "In this notebook we'll be using a fairly large batch size of 1024 just to make optimization run faster, but you could of course change this as you so choose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import TensorDataset, DataLoader\n",
    "train_dataset = TensorDataset(train_x, train_y)\n",
    "train_loader = DataLoader(train_dataset, batch_size=1024, shuffle=True)\n",
    "\n",
    "test_dataset = TensorDataset(test_x, test_y)\n",
    "test_loader = DataLoader(test_dataset, batch_size=1024, shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining the SVGP Model\n",
    "\n",
    "We now define the GP regression module that, intuitvely, will act as the final \"layer\" of our neural network. In this case, because we are doing variational inference and *not* exact inference, we will be using an `AbstractVariationalGP`. In this example, because we will be learning the inducing point locations, we'll be using a base `VariationalStrategy` with `learn_inducing_locations=True`.\n",
    "\n",
    "Because the feature extractor we defined above extracts two features, we'll need to define our grid bounds over two dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gpytorch.models import AbstractVariationalGP\n",
    "from gpytorch.variational import CholeskyVariationalDistribution\n",
    "from gpytorch.variational import WhitenedVariationalStrategy\n",
    "\n",
    "class GPModel(AbstractVariationalGP):\n",
    "    def __init__(self, inducing_points):\n",
    "        variational_distribution = CholeskyVariationalDistribution(inducing_points.size(0))\n",
    "        variational_strategy = WhitenedVariationalStrategy(self, inducing_points, variational_distribution, learn_inducing_locations=True)\n",
    "        super(GPModel, self).__init__(variational_strategy)\n",
    "        self.mean_module = gpytorch.means.ConstantMean()\n",
    "        self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel())\n",
    "        \n",
    "    def forward(self, x):\n",
    "        mean_x = self.mean_module(x)\n",
    "        covar_x = self.covar_module(x)\n",
    "        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)\n",
    "\n",
    "inducing_points = train_x[:500, :]\n",
    "model = GPModel(inducing_points=inducing_points).cuda()\n",
    "likelihood = gpytorch.likelihoods.GaussianLikelihood().cuda()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the Model\n",
    "\n",
    "The cell below trains the model above, learning both the hyperparameters of the Gaussian process **and** the parameters of the neural network in an end-to-end fashion using Type-II MLE.\n",
    "\n",
    "Unlike when using the exact GP marginal log likelihood, performing variational inference allows us to make use of stochastic optimization techniques. For this example, we'll do one epoch of training. Given the small size of the neural network relative to the size of the dataset, this should be sufficient to achieve comparable accuracy to what was observed in the DKL paper.\n",
    "\n",
    "The optimization loop differs from the one seen in our more simple tutorials in that it involves looping over both a number of training iterations (epochs) *and* minibatches of the data. However, the basic process is the same: for each minibatch, we forward through the model, compute the loss (the `VariationalMarginalLogLikelihood` or ELBO), call backwards, and do a step of optimization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 [0/340] - Loss: 284.605 [-284.605, -0.000, 0.000]\n",
      "Epoch 1 [1/340] - Loss: 407.885 [-407.884, 0.001, 0.000]\n",
      "Epoch 1 [2/340] - Loss: 266.743 [-266.743, 0.000, 0.000]\n",
      "Epoch 1 [3/340] - Loss: 314.733 [-314.733, 0.000, 0.000]\n",
      "Epoch 1 [4/340] - Loss: 354.599 [-354.598, 0.001, 0.000]\n",
      "Epoch 1 [5/340] - Loss: 313.988 [-313.988, 0.001, 0.000]\n",
      "Epoch 1 [6/340] - Loss: 269.362 [-269.362, 0.000, 0.000]\n",
      "Epoch 1 [7/340] - Loss: 251.600 [-251.600, -0.000, 0.000]\n",
      "Epoch 1 [8/340] - Loss: 278.105 [-278.105, 0.000, 0.000]\n",
      "Epoch 1 [9/340] - Loss: 254.841 [-254.841, 0.000, 0.000]\n",
      "Epoch 1 [10/340] - Loss: 288.696 [-288.696, 0.000, 0.000]\n",
      "Epoch 1 [11/340] - Loss: 236.707 [-236.707, 0.000, 0.000]\n",
      "Epoch 1 [12/340] - Loss: 230.295 [-230.295, 0.000, 0.000]\n",
      "Epoch 1 [13/340] - Loss: 212.733 [-212.733, 0.000, 0.000]\n",
      "Epoch 1 [14/340] - Loss: 238.513 [-238.513, 0.000, 0.000]\n",
      "Epoch 1 [15/340] - Loss: 241.217 [-241.217, 0.000, 0.000]\n",
      "Epoch 1 [16/340] - Loss: 214.024 [-214.024, 0.000, 0.000]\n",
      "Epoch 1 [17/340] - Loss: 255.499 [-255.499, 0.000, 0.000]\n",
      "Epoch 1 [18/340] - Loss: 242.679 [-242.679, 0.000, 0.000]\n",
      "Epoch 1 [19/340] - Loss: 221.330 [-221.330, 0.000, 0.000]\n",
      "Epoch 1 [20/340] - Loss: 230.084 [-230.084, 0.000, 0.000]\n",
      "Epoch 1 [21/340] - Loss: 195.431 [-195.431, 0.000, 0.000]\n",
      "Epoch 1 [22/340] - Loss: 222.145 [-222.145, 0.000, 0.000]\n",
      "Epoch 1 [23/340] - Loss: 218.169 [-218.168, 0.000, 0.000]\n",
      "Epoch 1 [24/340] - Loss: 218.322 [-218.321, 0.000, 0.000]\n",
      "Epoch 1 [25/340] - Loss: 208.488 [-208.487, 0.000, 0.000]\n",
      "Epoch 1 [26/340] - Loss: 208.791 [-208.790, 0.000, 0.000]\n",
      "Epoch 1 [27/340] - Loss: 242.118 [-242.118, 0.000, 0.000]\n",
      "Epoch 1 [28/340] - Loss: 211.672 [-211.671, 0.000, 0.000]\n",
      "Epoch 1 [29/340] - Loss: 213.223 [-213.222, 0.000, 0.000]\n",
      "Epoch 1 [30/340] - Loss: 215.091 [-215.090, 0.001, 0.000]\n",
      "Epoch 1 [31/340] - Loss: 218.898 [-218.898, 0.001, 0.000]\n",
      "Epoch 1 [32/340] - Loss: 196.652 [-196.651, 0.001, 0.000]\n",
      "Epoch 1 [33/340] - Loss: 181.427 [-181.426, 0.001, 0.000]\n",
      "Epoch 1 [34/340] - Loss: 196.068 [-196.067, 0.001, 0.000]\n",
      "Epoch 1 [35/340] - Loss: 212.519 [-212.518, 0.001, 0.000]\n",
      "Epoch 1 [36/340] - Loss: 196.792 [-196.791, 0.001, 0.000]\n",
      "Epoch 1 [37/340] - Loss: 191.424 [-191.423, 0.001, 0.000]\n",
      "Epoch 1 [38/340] - Loss: 218.270 [-218.269, 0.001, 0.000]\n",
      "Epoch 1 [39/340] - Loss: 203.168 [-203.167, 0.001, 0.000]\n",
      "Epoch 1 [40/340] - Loss: 211.636 [-211.635, 0.001, 0.000]\n",
      "Epoch 1 [41/340] - Loss: 188.339 [-188.338, 0.001, 0.000]\n",
      "Epoch 1 [42/340] - Loss: 169.502 [-169.501, 0.001, 0.000]\n",
      "Epoch 1 [43/340] - Loss: 201.711 [-201.710, 0.001, 0.000]\n",
      "Epoch 1 [44/340] - Loss: 193.040 [-193.039, 0.001, 0.000]\n",
      "Epoch 1 [45/340] - Loss: 178.395 [-178.393, 0.001, 0.000]\n",
      "Epoch 1 [46/340] - Loss: 183.074 [-183.073, 0.001, 0.000]\n",
      "Epoch 1 [47/340] - Loss: 196.607 [-196.605, 0.001, 0.000]\n",
      "Epoch 1 [48/340] - Loss: 197.626 [-197.625, 0.001, 0.000]\n",
      "Epoch 1 [49/340] - Loss: 170.022 [-170.020, 0.001, 0.000]\n",
      "Epoch 1 [50/340] - Loss: 171.350 [-171.349, 0.001, 0.000]\n",
      "Epoch 1 [51/340] - Loss: 204.937 [-204.935, 0.001, 0.000]\n",
      "Epoch 1 [52/340] - Loss: 190.182 [-190.181, 0.002, 0.000]\n",
      "Epoch 1 [53/340] - Loss: 181.296 [-181.295, 0.002, 0.000]\n",
      "Epoch 1 [54/340] - Loss: 165.575 [-165.573, 0.002, 0.000]\n",
      "Epoch 1 [55/340] - Loss: 177.188 [-177.186, 0.002, 0.000]\n",
      "Epoch 1 [56/340] - Loss: 162.593 [-162.591, 0.002, 0.000]\n",
      "Epoch 1 [57/340] - Loss: 178.402 [-178.400, 0.002, 0.000]\n",
      "Epoch 1 [58/340] - Loss: 178.004 [-178.002, 0.002, 0.000]\n",
      "Epoch 1 [59/340] - Loss: 167.633 [-167.631, 0.002, 0.000]\n",
      "Epoch 1 [60/340] - Loss: 163.924 [-163.922, 0.002, 0.000]\n",
      "Epoch 1 [61/340] - Loss: 149.788 [-149.786, 0.002, 0.000]\n",
      "Epoch 1 [62/340] - Loss: 154.860 [-154.857, 0.002, 0.000]\n",
      "Epoch 1 [63/340] - Loss: 187.472 [-187.469, 0.002, 0.000]\n",
      "Epoch 1 [64/340] - Loss: 153.802 [-153.799, 0.002, 0.000]\n",
      "Epoch 1 [65/340] - Loss: 170.994 [-170.992, 0.003, 0.000]\n",
      "Epoch 1 [66/340] - Loss: 159.923 [-159.921, 0.003, 0.000]\n",
      "Epoch 1 [67/340] - Loss: 175.249 [-175.246, 0.003, 0.000]\n",
      "Epoch 1 [68/340] - Loss: 155.151 [-155.148, 0.003, 0.000]\n",
      "Epoch 1 [69/340] - Loss: 163.742 [-163.739, 0.003, 0.000]\n",
      "Epoch 1 [70/340] - Loss: 150.713 [-150.710, 0.003, 0.000]\n",
      "Epoch 1 [71/340] - Loss: 155.996 [-155.993, 0.003, 0.000]\n",
      "Epoch 1 [72/340] - Loss: 166.343 [-166.340, 0.003, 0.000]\n",
      "Epoch 1 [73/340] - Loss: 173.412 [-173.408, 0.003, 0.000]\n",
      "Epoch 1 [74/340] - Loss: 141.471 [-141.467, 0.003, 0.000]\n",
      "Epoch 1 [75/340] - Loss: 155.533 [-155.529, 0.004, 0.000]\n",
      "Epoch 1 [76/340] - Loss: 143.602 [-143.598, 0.004, 0.000]\n",
      "Epoch 1 [77/340] - Loss: 149.557 [-149.553, 0.004, 0.000]\n",
      "Epoch 1 [78/340] - Loss: 148.940 [-148.936, 0.004, 0.000]\n",
      "Epoch 1 [79/340] - Loss: 159.048 [-159.044, 0.004, 0.000]\n",
      "Epoch 1 [80/340] - Loss: 137.916 [-137.912, 0.004, 0.000]\n",
      "Epoch 1 [81/340] - Loss: 147.184 [-147.179, 0.004, 0.000]\n",
      "Epoch 1 [82/340] - Loss: 149.096 [-149.092, 0.004, 0.000]\n",
      "Epoch 1 [83/340] - Loss: 132.778 [-132.774, 0.005, 0.000]\n",
      "Epoch 1 [84/340] - Loss: 148.329 [-148.324, 0.005, 0.000]\n",
      "Epoch 1 [85/340] - Loss: 148.233 [-148.228, 0.005, 0.000]\n",
      "Epoch 1 [86/340] - Loss: 136.019 [-136.014, 0.005, 0.000]\n",
      "Epoch 1 [87/340] - Loss: 142.694 [-142.689, 0.005, 0.000]\n",
      "Epoch 1 [88/340] - Loss: 154.479 [-154.474, 0.005, 0.000]\n",
      "Epoch 1 [89/340] - Loss: 157.358 [-157.353, 0.005, 0.000]\n",
      "Epoch 1 [90/340] - Loss: 150.447 [-150.441, 0.006, 0.000]\n",
      "Epoch 1 [91/340] - Loss: 141.485 [-141.479, 0.006, 0.000]\n",
      "Epoch 1 [92/340] - Loss: 134.669 [-134.663, 0.006, 0.000]\n",
      "Epoch 1 [93/340] - Loss: 137.915 [-137.909, 0.006, 0.000]\n",
      "Epoch 1 [94/340] - Loss: 134.577 [-134.570, 0.006, 0.000]\n",
      "Epoch 1 [95/340] - Loss: 136.917 [-136.910, 0.007, 0.000]\n",
      "Epoch 1 [96/340] - Loss: 139.852 [-139.846, 0.007, 0.000]\n",
      "Epoch 1 [97/340] - Loss: 130.827 [-130.820, 0.007, 0.000]\n",
      "Epoch 1 [98/340] - Loss: 122.095 [-122.088, 0.007, 0.000]\n",
      "Epoch 1 [99/340] - Loss: 135.648 [-135.640, 0.007, 0.000]\n",
      "Epoch 1 [100/340] - Loss: 125.961 [-125.954, 0.007, 0.000]\n",
      "Epoch 1 [101/340] - Loss: 137.866 [-137.859, 0.008, 0.000]\n",
      "Epoch 1 [102/340] - Loss: 128.404 [-128.396, 0.008, 0.000]\n",
      "Epoch 1 [103/340] - Loss: 149.411 [-149.403, 0.008, 0.000]\n",
      "Epoch 1 [104/340] - Loss: 137.718 [-137.710, 0.008, 0.000]\n",
      "Epoch 1 [105/340] - Loss: 128.840 [-128.832, 0.008, 0.000]\n",
      "Epoch 1 [106/340] - Loss: 120.813 [-120.805, 0.008, 0.000]\n",
      "Epoch 1 [107/340] - Loss: 138.833 [-138.825, 0.009, 0.000]\n",
      "Epoch 1 [108/340] - Loss: 134.063 [-134.054, 0.009, 0.000]\n",
      "Epoch 1 [109/340] - Loss: 122.921 [-122.912, 0.009, 0.000]\n",
      "Epoch 1 [110/340] - Loss: 130.113 [-130.104, 0.009, 0.000]\n",
      "Epoch 1 [111/340] - Loss: 132.207 [-132.198, 0.009, 0.000]\n",
      "Epoch 1 [112/340] - Loss: 120.904 [-120.894, 0.009, 0.000]\n",
      "Epoch 1 [113/340] - Loss: 120.676 [-120.666, 0.010, 0.000]\n",
      "Epoch 1 [114/340] - Loss: 126.008 [-125.998, 0.010, 0.000]\n",
      "Epoch 1 [115/340] - Loss: 130.670 [-130.660, 0.010, 0.000]\n",
      "Epoch 1 [116/340] - Loss: 114.772 [-114.761, 0.010, 0.000]\n",
      "Epoch 1 [117/340] - Loss: 119.130 [-119.120, 0.010, 0.000]\n",
      "Epoch 1 [118/340] - Loss: 116.937 [-116.927, 0.011, 0.000]\n",
      "Epoch 1 [119/340] - Loss: 124.997 [-124.986, 0.011, 0.000]\n",
      "Epoch 1 [120/340] - Loss: 110.194 [-110.183, 0.011, 0.000]\n",
      "Epoch 1 [121/340] - Loss: 117.811 [-117.800, 0.011, 0.000]\n",
      "Epoch 1 [122/340] - Loss: 112.518 [-112.507, 0.011, 0.000]\n",
      "Epoch 1 [123/340] - Loss: 125.268 [-125.257, 0.012, 0.000]\n",
      "Epoch 1 [124/340] - Loss: 120.532 [-120.521, 0.012, 0.000]\n",
      "Epoch 1 [125/340] - Loss: 124.577 [-124.565, 0.012, 0.000]\n",
      "Epoch 1 [126/340] - Loss: 101.105 [-101.093, 0.012, 0.000]\n",
      "Epoch 1 [127/340] - Loss: 105.848 [-105.835, 0.012, 0.000]\n",
      "Epoch 1 [128/340] - Loss: 117.697 [-117.684, 0.013, 0.000]\n",
      "Epoch 1 [129/340] - Loss: 125.651 [-125.639, 0.013, 0.000]\n",
      "Epoch 1 [130/340] - Loss: 119.657 [-119.644, 0.013, 0.000]\n",
      "Epoch 1 [131/340] - Loss: 110.837 [-110.824, 0.013, 0.000]\n",
      "Epoch 1 [132/340] - Loss: 115.313 [-115.299, 0.013, 0.000]\n",
      "Epoch 1 [133/340] - Loss: 109.737 [-109.723, 0.014, 0.000]\n",
      "Epoch 1 [134/340] - Loss: 104.708 [-104.694, 0.014, 0.000]\n",
      "Epoch 1 [135/340] - Loss: 108.345 [-108.330, 0.014, 0.000]\n",
      "Epoch 1 [136/340] - Loss: 98.066 [-98.051, 0.015, 0.000]\n",
      "Epoch 1 [137/340] - Loss: 107.812 [-107.797, 0.015, 0.000]\n",
      "Epoch 1 [138/340] - Loss: 119.069 [-119.054, 0.016, 0.000]\n",
      "Epoch 1 [139/340] - Loss: 116.237 [-116.221, 0.016, 0.000]\n",
      "Epoch 1 [140/340] - Loss: 111.332 [-111.316, 0.016, 0.000]\n",
      "Epoch 1 [141/340] - Loss: 104.349 [-104.332, 0.017, 0.000]\n",
      "Epoch 1 [142/340] - Loss: 88.443 [-88.426, 0.017, 0.000]\n",
      "Epoch 1 [143/340] - Loss: 103.385 [-103.368, 0.018, 0.000]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 [144/340] - Loss: 101.966 [-101.948, 0.018, 0.000]\n",
      "Epoch 1 [145/340] - Loss: 100.822 [-100.804, 0.018, 0.000]\n",
      "Epoch 1 [146/340] - Loss: 113.399 [-113.381, 0.018, 0.000]\n",
      "Epoch 1 [147/340] - Loss: 98.197 [-98.178, 0.019, 0.000]\n",
      "Epoch 1 [148/340] - Loss: 95.927 [-95.908, 0.019, 0.000]\n",
      "Epoch 1 [149/340] - Loss: 103.746 [-103.726, 0.020, 0.000]\n",
      "Epoch 1 [150/340] - Loss: 100.094 [-100.074, 0.020, 0.000]\n",
      "Epoch 1 [151/340] - Loss: 99.724 [-99.703, 0.021, 0.000]\n",
      "Epoch 1 [152/340] - Loss: 105.135 [-105.114, 0.021, 0.000]\n",
      "Epoch 1 [153/340] - Loss: 105.083 [-105.062, 0.022, 0.000]\n",
      "Epoch 1 [154/340] - Loss: 99.960 [-99.938, 0.022, 0.000]\n",
      "Epoch 1 [155/340] - Loss: 97.062 [-97.040, 0.022, 0.000]\n",
      "Epoch 1 [156/340] - Loss: 102.908 [-102.885, 0.023, 0.000]\n",
      "Epoch 1 [157/340] - Loss: 101.583 [-101.560, 0.023, 0.000]\n",
      "Epoch 1 [158/340] - Loss: 88.308 [-88.285, 0.024, 0.000]\n",
      "Epoch 1 [159/340] - Loss: 94.407 [-94.383, 0.024, 0.000]\n",
      "Epoch 1 [160/340] - Loss: 94.089 [-94.065, 0.024, 0.000]\n",
      "Epoch 1 [161/340] - Loss: 96.391 [-96.366, 0.025, 0.000]\n",
      "Epoch 1 [162/340] - Loss: 91.696 [-91.670, 0.026, 0.000]\n",
      "Epoch 1 [163/340] - Loss: 97.112 [-97.086, 0.026, 0.000]\n",
      "Epoch 1 [164/340] - Loss: 94.734 [-94.708, 0.026, 0.000]\n",
      "Epoch 1 [165/340] - Loss: 92.835 [-92.809, 0.026, 0.000]\n",
      "Epoch 1 [166/340] - Loss: 88.268 [-88.241, 0.026, 0.000]\n",
      "Epoch 1 [167/340] - Loss: 97.374 [-97.347, 0.027, 0.000]\n",
      "Epoch 1 [168/340] - Loss: 102.940 [-102.912, 0.028, 0.000]\n",
      "Epoch 1 [169/340] - Loss: 90.435 [-90.406, 0.029, 0.000]\n",
      "Epoch 1 [170/340] - Loss: 86.725 [-86.694, 0.031, 0.000]\n",
      "Epoch 1 [171/340] - Loss: 92.959 [-92.927, 0.032, 0.000]\n",
      "Epoch 1 [172/340] - Loss: 82.381 [-82.348, 0.033, 0.000]\n",
      "Epoch 1 [173/340] - Loss: 88.950 [-88.916, 0.034, 0.000]\n",
      "Epoch 1 [174/340] - Loss: 89.003 [-88.969, 0.034, 0.000]\n",
      "Epoch 1 [175/340] - Loss: 91.773 [-91.739, 0.034, 0.000]\n",
      "Epoch 1 [176/340] - Loss: 88.164 [-88.131, 0.033, 0.000]\n",
      "Epoch 1 [177/340] - Loss: 95.640 [-95.608, 0.032, 0.000]\n",
      "Epoch 1 [178/340] - Loss: 96.515 [-96.483, 0.033, 0.000]\n",
      "Epoch 1 [179/340] - Loss: 86.761 [-86.727, 0.034, 0.000]\n",
      "Epoch 1 [180/340] - Loss: 89.810 [-89.774, 0.035, 0.000]\n",
      "Epoch 1 [181/340] - Loss: 82.934 [-82.898, 0.036, 0.000]\n",
      "Epoch 1 [182/340] - Loss: 81.976 [-81.938, 0.038, 0.000]\n",
      "Epoch 1 [183/340] - Loss: 86.439 [-86.400, 0.039, 0.000]\n",
      "Epoch 1 [184/340] - Loss: 83.551 [-83.511, 0.040, 0.000]\n",
      "Epoch 1 [185/340] - Loss: 88.792 [-88.752, 0.041, 0.000]\n",
      "Epoch 1 [186/340] - Loss: 80.867 [-80.826, 0.041, 0.000]\n",
      "Epoch 1 [187/340] - Loss: 82.245 [-82.205, 0.040, 0.000]\n",
      "Epoch 1 [188/340] - Loss: 77.642 [-77.602, 0.040, 0.000]\n",
      "Epoch 1 [189/340] - Loss: 80.525 [-80.486, 0.040, 0.000]\n",
      "Epoch 1 [190/340] - Loss: 89.857 [-89.818, 0.039, 0.000]\n",
      "Epoch 1 [191/340] - Loss: 81.933 [-81.894, 0.039, 0.000]\n",
      "Epoch 1 [192/340] - Loss: 77.736 [-77.697, 0.039, 0.000]\n",
      "Epoch 1 [193/340] - Loss: 76.406 [-76.366, 0.040, 0.000]\n",
      "Epoch 1 [194/340] - Loss: 79.428 [-79.387, 0.041, 0.000]\n",
      "Epoch 1 [195/340] - Loss: 81.420 [-81.377, 0.043, 0.000]\n",
      "Epoch 1 [196/340] - Loss: 81.879 [-81.835, 0.044, 0.000]\n",
      "Epoch 1 [197/340] - Loss: 79.824 [-79.781, 0.043, 0.000]\n",
      "Epoch 1 [198/340] - Loss: 77.589 [-77.546, 0.042, 0.000]\n",
      "Epoch 1 [199/340] - Loss: 81.281 [-81.238, 0.042, 0.000]\n",
      "Epoch 1 [200/340] - Loss: 82.944 [-82.902, 0.042, 0.000]\n",
      "Epoch 1 [201/340] - Loss: 75.267 [-75.224, 0.043, 0.000]\n",
      "Epoch 1 [202/340] - Loss: 75.733 [-75.690, 0.043, 0.000]\n",
      "Epoch 1 [203/340] - Loss: 89.787 [-89.744, 0.043, 0.000]\n",
      "Epoch 1 [204/340] - Loss: 81.357 [-81.313, 0.044, 0.000]\n",
      "Epoch 1 [205/340] - Loss: 84.994 [-84.949, 0.045, 0.000]\n",
      "Epoch 1 [206/340] - Loss: 74.605 [-74.560, 0.046, 0.000]\n",
      "Epoch 1 [207/340] - Loss: 77.942 [-77.896, 0.046, 0.000]\n",
      "Epoch 1 [208/340] - Loss: 76.621 [-76.574, 0.047, 0.000]\n",
      "Epoch 1 [209/340] - Loss: 97.975 [-97.926, 0.048, 0.000]\n",
      "Epoch 1 [210/340] - Loss: 69.899 [-69.850, 0.049, 0.000]\n",
      "Epoch 1 [211/340] - Loss: 83.126 [-83.076, 0.050, 0.000]\n",
      "Epoch 1 [212/340] - Loss: 86.309 [-86.261, 0.048, 0.000]\n",
      "Epoch 1 [213/340] - Loss: 78.918 [-78.870, 0.048, 0.000]\n",
      "Epoch 1 [214/340] - Loss: 77.152 [-77.105, 0.047, 0.000]\n",
      "Epoch 1 [215/340] - Loss: 72.817 [-72.770, 0.047, 0.000]\n",
      "Epoch 1 [216/340] - Loss: 73.060 [-73.012, 0.048, 0.000]\n",
      "Epoch 1 [217/340] - Loss: 76.522 [-76.473, 0.049, 0.000]\n",
      "Epoch 1 [218/340] - Loss: 71.380 [-71.331, 0.049, 0.000]\n",
      "Epoch 1 [219/340] - Loss: 79.210 [-79.161, 0.049, 0.000]\n",
      "Epoch 1 [220/340] - Loss: 72.317 [-72.268, 0.049, 0.000]\n",
      "Epoch 1 [221/340] - Loss: 69.445 [-69.396, 0.050, 0.000]\n",
      "Epoch 1 [222/340] - Loss: 77.372 [-77.321, 0.051, 0.000]\n",
      "Epoch 1 [223/340] - Loss: 76.444 [-76.392, 0.052, 0.000]\n",
      "Epoch 1 [224/340] - Loss: 72.151 [-72.096, 0.055, 0.000]\n",
      "Epoch 1 [225/340] - Loss: 70.297 [-70.240, 0.057, 0.000]\n",
      "Epoch 1 [226/340] - Loss: 76.991 [-76.930, 0.061, 0.000]\n",
      "Epoch 1 [227/340] - Loss: 69.771 [-69.710, 0.062, 0.000]\n",
      "Epoch 1 [228/340] - Loss: 70.927 [-70.865, 0.062, 0.000]\n",
      "Epoch 1 [229/340] - Loss: 77.032 [-76.967, 0.065, 0.000]\n",
      "Epoch 1 [230/340] - Loss: 67.278 [-67.213, 0.065, 0.000]\n",
      "Epoch 1 [231/340] - Loss: 68.727 [-68.663, 0.064, 0.000]\n",
      "Epoch 1 [232/340] - Loss: 64.965 [-64.906, 0.059, 0.000]\n",
      "Epoch 1 [233/340] - Loss: 68.797 [-68.743, 0.054, 0.000]\n",
      "Epoch 1 [234/340] - Loss: 78.722 [-78.671, 0.051, 0.000]\n",
      "Epoch 1 [235/340] - Loss: 72.708 [-72.657, 0.050, 0.000]\n",
      "Epoch 1 [236/340] - Loss: 71.465 [-71.414, 0.051, 0.000]\n",
      "Epoch 1 [237/340] - Loss: 61.804 [-61.751, 0.053, 0.000]\n",
      "Epoch 1 [238/340] - Loss: 74.555 [-74.499, 0.057, 0.000]\n",
      "Epoch 1 [239/340] - Loss: 73.761 [-73.699, 0.062, 0.000]\n",
      "Epoch 1 [240/340] - Loss: 72.113 [-72.046, 0.067, 0.000]\n",
      "Epoch 1 [241/340] - Loss: 67.209 [-67.143, 0.066, 0.000]\n",
      "Epoch 1 [242/340] - Loss: 66.605 [-66.543, 0.062, 0.000]\n",
      "Epoch 1 [243/340] - Loss: 70.647 [-70.590, 0.057, 0.000]\n",
      "Epoch 1 [244/340] - Loss: 73.025 [-72.973, 0.052, 0.000]\n",
      "Epoch 1 [245/340] - Loss: 70.849 [-70.799, 0.050, 0.000]\n",
      "Epoch 1 [246/340] - Loss: 72.962 [-72.912, 0.050, 0.000]\n",
      "Epoch 1 [247/340] - Loss: 68.337 [-68.284, 0.053, 0.000]\n",
      "Epoch 1 [248/340] - Loss: 74.196 [-74.139, 0.057, 0.000]\n",
      "Epoch 1 [249/340] - Loss: 58.703 [-58.639, 0.063, 0.000]\n",
      "Epoch 1 [250/340] - Loss: 73.486 [-73.416, 0.070, 0.000]\n",
      "Epoch 1 [251/340] - Loss: 61.754 [-61.681, 0.074, 0.000]\n",
      "Epoch 1 [252/340] - Loss: 69.172 [-69.094, 0.078, 0.000]\n",
      "Epoch 1 [253/340] - Loss: 59.875 [-59.799, 0.076, 0.000]\n",
      "Epoch 1 [254/340] - Loss: 59.212 [-59.136, 0.076, 0.000]\n",
      "Epoch 1 [255/340] - Loss: 70.070 [-69.996, 0.074, 0.000]\n",
      "Epoch 1 [256/340] - Loss: 65.692 [-65.624, 0.068, 0.000]\n",
      "Epoch 1 [257/340] - Loss: 63.456 [-63.393, 0.063, 0.000]\n",
      "Epoch 1 [258/340] - Loss: 68.850 [-68.792, 0.058, 0.000]\n",
      "Epoch 1 [259/340] - Loss: 69.800 [-69.744, 0.055, 0.000]\n",
      "Epoch 1 [260/340] - Loss: 68.637 [-68.582, 0.054, 0.000]\n",
      "Epoch 1 [261/340] - Loss: 66.691 [-66.637, 0.054, 0.000]\n",
      "Epoch 1 [262/340] - Loss: 72.166 [-72.110, 0.055, 0.000]\n",
      "Epoch 1 [263/340] - Loss: 70.571 [-70.514, 0.058, 0.000]\n",
      "Epoch 1 [264/340] - Loss: 64.072 [-64.011, 0.061, 0.000]\n",
      "Epoch 1 [265/340] - Loss: 72.019 [-71.955, 0.064, 0.000]\n",
      "Epoch 1 [266/340] - Loss: 74.041 [-73.973, 0.068, 0.000]\n",
      "Epoch 1 [267/340] - Loss: 70.072 [-70.000, 0.072, 0.000]\n",
      "Epoch 1 [268/340] - Loss: 64.511 [-64.435, 0.076, 0.000]\n",
      "Epoch 1 [269/340] - Loss: 67.799 [-67.720, 0.079, 0.000]\n",
      "Epoch 1 [270/340] - Loss: 64.785 [-64.707, 0.078, 0.000]\n",
      "Epoch 1 [271/340] - Loss: 66.473 [-66.397, 0.076, 0.000]\n",
      "Epoch 1 [272/340] - Loss: 69.081 [-69.008, 0.072, 0.000]\n",
      "Epoch 1 [273/340] - Loss: 64.715 [-64.649, 0.067, 0.000]\n",
      "Epoch 1 [274/340] - Loss: 63.172 [-63.111, 0.060, 0.000]\n",
      "Epoch 1 [275/340] - Loss: 63.665 [-63.608, 0.057, 0.000]\n",
      "Epoch 1 [276/340] - Loss: 69.076 [-69.021, 0.055, 0.000]\n",
      "Epoch 1 [277/340] - Loss: 64.851 [-64.796, 0.055, 0.000]\n",
      "Epoch 1 [278/340] - Loss: 75.071 [-75.012, 0.059, 0.000]\n",
      "Epoch 1 [279/340] - Loss: 61.335 [-61.270, 0.065, 0.000]\n",
      "Epoch 1 [280/340] - Loss: 64.243 [-64.172, 0.071, 0.000]\n",
      "Epoch 1 [281/340] - Loss: 67.824 [-67.747, 0.077, 0.000]\n",
      "Epoch 1 [282/340] - Loss: 64.954 [-64.877, 0.078, 0.000]\n",
      "Epoch 1 [283/340] - Loss: 59.181 [-59.104, 0.077, 0.000]\n",
      "Epoch 1 [284/340] - Loss: 65.852 [-65.770, 0.082, 0.000]\n",
      "Epoch 1 [285/340] - Loss: 64.515 [-64.425, 0.091, 0.000]\n",
      "Epoch 1 [286/340] - Loss: 61.058 [-60.953, 0.105, 0.000]\n",
      "Epoch 1 [287/340] - Loss: 68.711 [-68.593, 0.118, 0.000]\n",
      "Epoch 1 [288/340] - Loss: 60.719 [-60.596, 0.123, 0.000]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1 [289/340] - Loss: 71.807 [-71.706, 0.102, 0.000]\n",
      "Epoch 1 [290/340] - Loss: 67.643 [-67.565, 0.078, 0.000]\n",
      "Epoch 1 [291/340] - Loss: 72.273 [-72.210, 0.062, 0.000]\n",
      "Epoch 1 [292/340] - Loss: 60.988 [-60.935, 0.054, 0.000]\n",
      "Epoch 1 [293/340] - Loss: 67.555 [-67.505, 0.050, 0.000]\n",
      "Epoch 1 [294/340] - Loss: 69.092 [-69.041, 0.051, 0.000]\n",
      "Epoch 1 [295/340] - Loss: 63.161 [-63.107, 0.054, 0.000]\n",
      "Epoch 1 [296/340] - Loss: 64.705 [-64.643, 0.062, 0.000]\n",
      "Epoch 1 [297/340] - Loss: 64.937 [-64.869, 0.068, 0.000]\n",
      "Epoch 1 [298/340] - Loss: 65.062 [-64.986, 0.076, 0.000]\n",
      "Epoch 1 [299/340] - Loss: 60.561 [-60.472, 0.089, 0.000]\n",
      "Epoch 1 [300/340] - Loss: 56.765 [-56.669, 0.096, 0.000]\n",
      "Epoch 1 [301/340] - Loss: 64.428 [-64.329, 0.098, 0.000]\n",
      "Epoch 1 [302/340] - Loss: 62.534 [-62.425, 0.109, 0.000]\n",
      "Epoch 1 [303/340] - Loss: 68.426 [-68.322, 0.105, 0.000]\n",
      "Epoch 1 [304/340] - Loss: 60.936 [-60.843, 0.093, 0.000]\n",
      "Epoch 1 [305/340] - Loss: 57.165 [-57.073, 0.092, 0.000]\n",
      "Epoch 1 [306/340] - Loss: 57.990 [-57.905, 0.085, 0.000]\n",
      "Epoch 1 [307/340] - Loss: 63.727 [-63.646, 0.081, 0.000]\n",
      "Epoch 1 [308/340] - Loss: 60.071 [-59.996, 0.075, 0.000]\n",
      "Epoch 1 [309/340] - Loss: 61.807 [-61.738, 0.069, 0.000]\n",
      "Epoch 1 [310/340] - Loss: 68.573 [-68.508, 0.065, 0.000]\n",
      "Epoch 1 [311/340] - Loss: 61.967 [-61.904, 0.063, 0.000]\n",
      "Epoch 1 [312/340] - Loss: 61.777 [-61.713, 0.063, 0.000]\n",
      "Epoch 1 [313/340] - Loss: 60.068 [-60.002, 0.066, 0.000]\n",
      "Epoch 1 [314/340] - Loss: 69.908 [-69.838, 0.070, 0.000]\n",
      "Epoch 1 [315/340] - Loss: 56.727 [-56.653, 0.074, 0.000]\n",
      "Epoch 1 [316/340] - Loss: 61.807 [-61.726, 0.080, 0.000]\n",
      "Epoch 1 [317/340] - Loss: 58.432 [-58.346, 0.086, 0.000]\n",
      "Epoch 1 [318/340] - Loss: 64.943 [-64.853, 0.090, 0.000]\n",
      "Epoch 1 [319/340] - Loss: 65.473 [-65.383, 0.090, 0.000]\n",
      "Epoch 1 [320/340] - Loss: 67.005 [-66.912, 0.092, 0.000]\n",
      "Epoch 1 [321/340] - Loss: 64.741 [-64.647, 0.094, 0.000]\n",
      "Epoch 1 [322/340] - Loss: 57.367 [-57.274, 0.093, 0.000]\n",
      "Epoch 1 [323/340] - Loss: 60.593 [-60.505, 0.088, 0.000]\n",
      "Epoch 1 [324/340] - Loss: 61.021 [-60.937, 0.084, 0.000]\n",
      "Epoch 1 [325/340] - Loss: 60.270 [-60.190, 0.079, 0.000]\n",
      "Epoch 1 [326/340] - Loss: 64.620 [-64.546, 0.073, 0.000]\n",
      "Epoch 1 [327/340] - Loss: 54.390 [-54.322, 0.068, 0.000]\n",
      "Epoch 1 [328/340] - Loss: 60.190 [-60.124, 0.065, 0.000]\n",
      "Epoch 1 [329/340] - Loss: 59.332 [-59.267, 0.065, 0.000]\n",
      "Epoch 1 [330/340] - Loss: 59.168 [-59.099, 0.068, 0.000]\n",
      "Epoch 1 [331/340] - Loss: 60.081 [-60.007, 0.074, 0.000]\n",
      "Epoch 1 [332/340] - Loss: 59.680 [-59.599, 0.081, 0.000]\n",
      "Epoch 1 [333/340] - Loss: 57.490 [-57.403, 0.087, 0.000]\n",
      "Epoch 1 [334/340] - Loss: 59.383 [-59.292, 0.091, 0.000]\n",
      "Epoch 1 [335/340] - Loss: 56.318 [-56.224, 0.094, 0.000]\n",
      "Epoch 1 [336/340] - Loss: 58.664 [-58.569, 0.095, 0.000]\n",
      "Epoch 1 [337/340] - Loss: 62.906 [-62.809, 0.097, 0.000]\n",
      "Epoch 1 [338/340] - Loss: 60.586 [-60.490, 0.096, 0.000]\n",
      "Epoch 1 [339/340] - Loss: 67.464 [-67.371, 0.094, 0.000]\n",
      "Epoch 2 [0/340] - Loss: 52.798 [-52.708, 0.091, 0.000]\n",
      "Epoch 2 [1/340] - Loss: 64.913 [-64.825, 0.088, 0.000]\n",
      "Epoch 2 [2/340] - Loss: 60.784 [-60.700, 0.084, 0.000]\n",
      "Epoch 2 [3/340] - Loss: 63.198 [-63.114, 0.083, 0.000]\n",
      "Epoch 2 [4/340] - Loss: 58.235 [-58.153, 0.082, 0.000]\n",
      "Epoch 2 [5/340] - Loss: 54.206 [-54.123, 0.083, 0.000]\n",
      "Epoch 2 [6/340] - Loss: 56.694 [-56.611, 0.083, 0.000]\n",
      "Epoch 2 [7/340] - Loss: 64.839 [-64.757, 0.083, 0.000]\n",
      "Epoch 2 [8/340] - Loss: 56.754 [-56.675, 0.080, 0.000]\n",
      "Epoch 2 [9/340] - Loss: 59.444 [-59.366, 0.077, 0.000]\n",
      "Epoch 2 [10/340] - Loss: 54.852 [-54.776, 0.076, 0.000]\n",
      "Epoch 2 [11/340] - Loss: 57.280 [-57.204, 0.076, 0.000]\n",
      "Epoch 2 [12/340] - Loss: 52.118 [-52.042, 0.076, 0.000]\n",
      "Epoch 2 [13/340] - Loss: 53.720 [-53.643, 0.077, 0.000]\n",
      "Epoch 2 [14/340] - Loss: 57.832 [-57.753, 0.079, 0.000]\n",
      "Epoch 2 [15/340] - Loss: 52.920 [-52.837, 0.083, 0.000]\n",
      "Epoch 2 [16/340] - Loss: 57.574 [-57.488, 0.086, 0.000]\n",
      "Epoch 2 [17/340] - Loss: 58.461 [-58.374, 0.087, 0.000]\n",
      "Epoch 2 [18/340] - Loss: 50.309 [-50.223, 0.086, 0.000]\n",
      "Epoch 2 [19/340] - Loss: 58.974 [-58.888, 0.086, 0.000]\n",
      "Epoch 2 [20/340] - Loss: 56.523 [-56.439, 0.084, 0.000]\n",
      "Epoch 2 [21/340] - Loss: 54.918 [-54.838, 0.080, 0.000]\n",
      "Epoch 2 [22/340] - Loss: 52.925 [-52.846, 0.078, 0.000]\n",
      "Epoch 2 [23/340] - Loss: 54.927 [-54.850, 0.077, 0.000]\n",
      "Epoch 2 [24/340] - Loss: 49.624 [-49.549, 0.075, 0.000]\n",
      "Epoch 2 [25/340] - Loss: 57.675 [-57.600, 0.075, 0.000]\n",
      "Epoch 2 [26/340] - Loss: 63.746 [-63.670, 0.077, 0.000]\n",
      "Epoch 2 [27/340] - Loss: 58.099 [-58.023, 0.076, 0.000]\n",
      "Epoch 2 [28/340] - Loss: 51.028 [-50.949, 0.079, 0.000]\n",
      "Epoch 2 [29/340] - Loss: 56.101 [-56.017, 0.084, 0.000]\n",
      "Epoch 2 [30/340] - Loss: 62.494 [-62.409, 0.085, 0.000]\n",
      "Epoch 2 [31/340] - Loss: 53.755 [-53.664, 0.091, 0.000]\n",
      "Epoch 2 [32/340] - Loss: 60.255 [-60.159, 0.096, 0.000]\n",
      "Epoch 2 [33/340] - Loss: 64.930 [-64.824, 0.106, 0.000]\n",
      "Epoch 2 [34/340] - Loss: 66.986 [-66.872, 0.114, 0.000]\n",
      "Epoch 2 [35/340] - Loss: 54.800 [-54.675, 0.125, 0.000]\n",
      "Epoch 2 [36/340] - Loss: 61.267 [-61.146, 0.121, 0.000]\n",
      "Epoch 2 [37/340] - Loss: 60.100 [-59.990, 0.110, 0.000]\n",
      "Epoch 2 [38/340] - Loss: 54.972 [-54.876, 0.095, 0.000]\n",
      "Epoch 2 [39/340] - Loss: 57.102 [-57.017, 0.085, 0.000]\n",
      "Epoch 2 [40/340] - Loss: 50.397 [-50.318, 0.079, 0.000]\n",
      "Epoch 2 [41/340] - Loss: 53.367 [-53.291, 0.077, 0.000]\n",
      "Epoch 2 [42/340] - Loss: 57.253 [-57.177, 0.076, 0.000]\n",
      "Epoch 2 [43/340] - Loss: 56.904 [-56.827, 0.077, 0.000]\n",
      "Epoch 2 [44/340] - Loss: 54.084 [-54.004, 0.080, 0.000]\n",
      "Epoch 2 [45/340] - Loss: 54.064 [-53.981, 0.083, 0.000]\n",
      "Epoch 2 [46/340] - Loss: 61.756 [-61.670, 0.085, 0.000]\n",
      "Epoch 2 [47/340] - Loss: 51.394 [-51.308, 0.086, 0.000]\n",
      "Epoch 2 [48/340] - Loss: 51.343 [-51.257, 0.085, 0.000]\n",
      "Epoch 2 [49/340] - Loss: 56.634 [-56.551, 0.083, 0.000]\n",
      "Epoch 2 [50/340] - Loss: 55.814 [-55.733, 0.082, 0.000]\n",
      "Epoch 2 [51/340] - Loss: 50.535 [-50.453, 0.082, 0.000]\n",
      "Epoch 2 [52/340] - Loss: 55.188 [-55.106, 0.082, 0.000]\n",
      "Epoch 2 [53/340] - Loss: 58.593 [-58.511, 0.082, 0.000]\n",
      "Epoch 2 [54/340] - Loss: 51.696 [-51.610, 0.085, 0.000]\n",
      "Epoch 2 [55/340] - Loss: 61.259 [-61.167, 0.092, 0.000]\n",
      "Epoch 2 [56/340] - Loss: 56.646 [-56.542, 0.104, 0.000]\n",
      "Epoch 2 [57/340] - Loss: 60.211 [-60.093, 0.118, 0.000]\n",
      "Epoch 2 [58/340] - Loss: 52.777 [-52.656, 0.121, 0.000]\n",
      "Epoch 2 [59/340] - Loss: 58.320 [-58.205, 0.116, 0.000]\n",
      "Epoch 2 [60/340] - Loss: 68.541 [-68.439, 0.102, 0.000]\n",
      "Epoch 2 [61/340] - Loss: 50.701 [-50.615, 0.086, 0.000]\n",
      "Epoch 2 [62/340] - Loss: 60.139 [-60.064, 0.075, 0.000]\n",
      "Epoch 2 [63/340] - Loss: 57.913 [-57.841, 0.073, 0.000]\n",
      "Epoch 2 [64/340] - Loss: 58.034 [-57.957, 0.077, 0.000]\n",
      "Epoch 2 [65/340] - Loss: 55.357 [-55.274, 0.083, 0.000]\n",
      "Epoch 2 [66/340] - Loss: 58.546 [-58.454, 0.092, 0.000]\n",
      "Epoch 2 [67/340] - Loss: 59.439 [-59.338, 0.100, 0.000]\n",
      "Epoch 2 [68/340] - Loss: 57.734 [-57.626, 0.108, 0.000]\n",
      "Epoch 2 [69/340] - Loss: 61.574 [-61.452, 0.123, 0.000]\n",
      "Epoch 2 [70/340] - Loss: 70.613 [-70.457, 0.156, 0.000]\n",
      "Epoch 2 [71/340] - Loss: 73.107 [-72.928, 0.179, 0.000]\n",
      "Epoch 2 [72/340] - Loss: 70.037 [-69.856, 0.180, 0.000]\n",
      "Epoch 2 [73/340] - Loss: 57.165 [-56.990, 0.175, 0.000]\n",
      "Epoch 2 [74/340] - Loss: 76.107 [-75.945, 0.163, 0.000]\n",
      "Epoch 2 [75/340] - Loss: 51.968 [-51.819, 0.149, 0.000]\n",
      "Epoch 2 [76/340] - Loss: 62.767 [-62.631, 0.136, 0.000]\n",
      "Epoch 2 [77/340] - Loss: 57.588 [-57.465, 0.122, 0.000]\n",
      "Epoch 2 [78/340] - Loss: 61.080 [-60.967, 0.113, 0.000]\n",
      "Epoch 2 [79/340] - Loss: 56.502 [-56.397, 0.105, 0.000]\n",
      "Epoch 2 [80/340] - Loss: 53.130 [-53.026, 0.103, 0.000]\n",
      "Epoch 2 [81/340] - Loss: 61.350 [-61.246, 0.104, 0.000]\n",
      "Epoch 2 [82/340] - Loss: 60.262 [-60.153, 0.109, 0.000]\n",
      "Epoch 2 [83/340] - Loss: 55.002 [-54.889, 0.114, 0.000]\n",
      "Epoch 2 [84/340] - Loss: 52.158 [-52.038, 0.120, 0.000]\n",
      "Epoch 2 [85/340] - Loss: 54.619 [-54.493, 0.126, 0.000]\n",
      "Epoch 2 [86/340] - Loss: 51.591 [-51.453, 0.138, 0.000]\n",
      "Epoch 2 [87/340] - Loss: 53.092 [-52.947, 0.145, 0.000]\n",
      "Epoch 2 [88/340] - Loss: 54.478 [-54.323, 0.155, 0.000]\n",
      "Epoch 2 [89/340] - Loss: 49.922 [-49.769, 0.153, 0.000]\n",
      "Epoch 2 [90/340] - Loss: 59.006 [-58.864, 0.143, 0.000]\n",
      "Epoch 2 [91/340] - Loss: 51.340 [-51.210, 0.130, 0.000]\n",
      "Epoch 2 [92/340] - Loss: 55.093 [-54.976, 0.117, 0.000]\n",
      "Epoch 2 [93/340] - Loss: 52.123 [-52.015, 0.108, 0.000]\n",
      "Epoch 2 [94/340] - Loss: 55.818 [-55.717, 0.102, 0.000]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 2 [95/340] - Loss: 50.599 [-50.499, 0.100, 0.000]\n",
      "Epoch 2 [96/340] - Loss: 61.430 [-61.327, 0.102, 0.000]\n",
      "Epoch 2 [97/340] - Loss: 52.454 [-52.350, 0.104, 0.000]\n",
      "Epoch 2 [98/340] - Loss: 57.813 [-57.703, 0.110, 0.000]\n",
      "Epoch 2 [99/340] - Loss: 54.270 [-54.157, 0.113, 0.000]\n",
      "Epoch 2 [100/340] - Loss: 47.857 [-47.741, 0.116, 0.000]\n",
      "Epoch 2 [101/340] - Loss: 51.680 [-51.559, 0.121, 0.000]\n",
      "Epoch 2 [102/340] - Loss: 49.377 [-49.252, 0.125, 0.000]\n",
      "Epoch 2 [103/340] - Loss: 49.895 [-49.764, 0.131, 0.000]\n",
      "Epoch 2 [104/340] - Loss: 48.714 [-48.577, 0.137, 0.000]\n",
      "Epoch 2 [105/340] - Loss: 44.537 [-44.395, 0.142, 0.000]\n",
      "Epoch 2 [106/340] - Loss: 49.458 [-49.316, 0.141, 0.000]\n",
      "Epoch 2 [107/340] - Loss: 48.410 [-48.270, 0.139, 0.000]\n",
      "Epoch 2 [108/340] - Loss: 56.088 [-55.952, 0.136, 0.000]\n",
      "Epoch 2 [109/340] - Loss: 50.461 [-50.328, 0.133, 0.000]\n",
      "Epoch 2 [110/340] - Loss: 55.284 [-55.155, 0.129, 0.000]\n",
      "Epoch 2 [111/340] - Loss: 48.117 [-47.995, 0.122, 0.000]\n",
      "Epoch 2 [112/340] - Loss: 48.796 [-48.680, 0.116, 0.000]\n",
      "Epoch 2 [113/340] - Loss: 53.400 [-53.288, 0.112, 0.000]\n",
      "Epoch 2 [114/340] - Loss: 58.465 [-58.361, 0.104, 0.000]\n",
      "Epoch 2 [115/340] - Loss: 48.608 [-48.509, 0.099, 0.000]\n",
      "Epoch 2 [116/340] - Loss: 48.664 [-48.567, 0.097, 0.000]\n",
      "Epoch 2 [117/340] - Loss: 46.038 [-45.942, 0.096, 0.000]\n",
      "Epoch 2 [118/340] - Loss: 51.139 [-51.043, 0.097, 0.000]\n",
      "Epoch 2 [119/340] - Loss: 51.422 [-51.324, 0.098, 0.000]\n",
      "Epoch 2 [120/340] - Loss: 47.391 [-47.290, 0.100, 0.000]\n",
      "Epoch 2 [121/340] - Loss: 50.118 [-50.016, 0.102, 0.000]\n",
      "Epoch 2 [122/340] - Loss: 44.372 [-44.268, 0.103, 0.000]\n",
      "Epoch 2 [123/340] - Loss: 42.744 [-42.639, 0.105, 0.000]\n",
      "Epoch 2 [124/340] - Loss: 46.832 [-46.725, 0.107, 0.000]\n",
      "Epoch 2 [125/340] - Loss: 56.179 [-56.068, 0.111, 0.000]\n",
      "Epoch 2 [126/340] - Loss: 47.780 [-47.671, 0.108, 0.000]\n",
      "Epoch 2 [127/340] - Loss: 47.777 [-47.670, 0.108, 0.000]\n",
      "Epoch 2 [128/340] - Loss: 53.653 [-53.544, 0.109, 0.000]\n",
      "Epoch 2 [129/340] - Loss: 47.240 [-47.132, 0.107, 0.000]\n",
      "Epoch 2 [130/340] - Loss: 51.401 [-51.294, 0.107, 0.000]\n",
      "Epoch 2 [131/340] - Loss: 45.279 [-45.171, 0.107, 0.000]\n",
      "Epoch 2 [132/340] - Loss: 46.969 [-46.857, 0.111, 0.000]\n",
      "Epoch 2 [133/340] - Loss: 54.870 [-54.753, 0.117, 0.000]\n",
      "Epoch 2 [134/340] - Loss: 52.343 [-52.218, 0.125, 0.000]\n",
      "Epoch 2 [135/340] - Loss: 45.783 [-45.654, 0.129, 0.000]\n",
      "Epoch 2 [136/340] - Loss: 51.022 [-50.887, 0.134, 0.000]\n",
      "Epoch 2 [137/340] - Loss: 53.078 [-52.936, 0.142, 0.000]\n",
      "Epoch 2 [138/340] - Loss: 51.479 [-51.334, 0.145, 0.000]\n",
      "Epoch 2 [139/340] - Loss: 47.604 [-47.463, 0.141, 0.000]\n",
      "Epoch 2 [140/340] - Loss: 53.238 [-53.099, 0.140, 0.000]\n",
      "Epoch 2 [141/340] - Loss: 51.706 [-51.571, 0.135, 0.000]\n",
      "Epoch 2 [142/340] - Loss: 56.260 [-56.138, 0.123, 0.000]\n",
      "Epoch 2 [143/340] - Loss: 45.326 [-45.216, 0.110, 0.000]\n",
      "Epoch 2 [144/340] - Loss: 54.241 [-54.138, 0.103, 0.000]\n",
      "Epoch 2 [145/340] - Loss: 44.950 [-44.854, 0.096, 0.000]\n",
      "Epoch 2 [146/340] - Loss: 43.742 [-43.648, 0.094, 0.000]\n",
      "Epoch 2 [147/340] - Loss: 45.935 [-45.840, 0.095, 0.000]\n",
      "Epoch 2 [148/340] - Loss: 47.721 [-47.621, 0.099, 0.000]\n",
      "Epoch 2 [149/340] - Loss: 45.030 [-44.922, 0.108, 0.000]\n",
      "Epoch 2 [150/340] - Loss: 52.395 [-52.279, 0.116, 0.000]\n",
      "Epoch 2 [151/340] - Loss: 53.678 [-53.550, 0.128, 0.000]\n",
      "Epoch 2 [152/340] - Loss: 51.045 [-50.908, 0.137, 0.000]\n",
      "Epoch 2 [153/340] - Loss: 47.954 [-47.807, 0.147, 0.000]\n",
      "Epoch 2 [154/340] - Loss: 52.448 [-52.302, 0.146, 0.000]\n",
      "Epoch 2 [155/340] - Loss: 70.695 [-70.537, 0.158, 0.000]\n",
      "Epoch 2 [156/340] - Loss: 50.344 [-50.191, 0.154, 0.000]\n",
      "Epoch 2 [157/340] - Loss: 54.765 [-54.607, 0.159, 0.000]\n",
      "Epoch 2 [158/340] - Loss: 66.402 [-66.247, 0.155, 0.000]\n",
      "Epoch 2 [159/340] - Loss: 51.394 [-51.242, 0.153, 0.000]\n",
      "Epoch 2 [160/340] - Loss: 47.643 [-47.497, 0.146, 0.000]\n",
      "Epoch 2 [161/340] - Loss: 46.817 [-46.675, 0.142, 0.000]\n",
      "Epoch 2 [162/340] - Loss: 49.137 [-49.000, 0.137, 0.000]\n",
      "Epoch 2 [163/340] - Loss: 52.504 [-52.376, 0.129, 0.000]\n",
      "Epoch 2 [164/340] - Loss: 58.865 [-58.739, 0.126, 0.000]\n",
      "Epoch 2 [165/340] - Loss: 59.828 [-59.713, 0.115, 0.000]\n",
      "Epoch 2 [166/340] - Loss: 48.528 [-48.421, 0.108, 0.000]\n",
      "Epoch 2 [167/340] - Loss: 50.350 [-50.248, 0.102, 0.000]\n",
      "Epoch 2 [168/340] - Loss: 52.892 [-52.785, 0.106, 0.000]\n",
      "Epoch 2 [169/340] - Loss: 48.012 [-47.908, 0.104, 0.000]\n",
      "Epoch 2 [170/340] - Loss: 49.402 [-49.294, 0.108, 0.000]\n",
      "Epoch 2 [171/340] - Loss: 45.827 [-45.714, 0.114, 0.000]\n",
      "Epoch 2 [172/340] - Loss: 44.929 [-44.811, 0.118, 0.000]\n",
      "Epoch 2 [173/340] - Loss: 51.114 [-50.990, 0.125, 0.000]\n",
      "Epoch 2 [174/340] - Loss: 52.272 [-52.138, 0.134, 0.000]\n",
      "Epoch 2 [175/340] - Loss: 43.047 [-42.907, 0.140, 0.000]\n",
      "Epoch 2 [176/340] - Loss: 45.224 [-45.073, 0.151, 0.000]\n",
      "Epoch 2 [177/340] - Loss: 50.601 [-50.434, 0.167, 0.000]\n",
      "Epoch 2 [178/340] - Loss: 49.416 [-49.240, 0.176, 0.000]\n",
      "Epoch 2 [179/340] - Loss: 54.722 [-54.538, 0.184, 0.000]\n",
      "Epoch 2 [180/340] - Loss: 55.935 [-55.758, 0.177, 0.000]\n",
      "Epoch 2 [181/340] - Loss: 49.812 [-49.645, 0.167, 0.000]\n",
      "Epoch 2 [182/340] - Loss: 46.207 [-46.047, 0.160, 0.000]\n",
      "Epoch 2 [183/340] - Loss: 47.638 [-47.482, 0.156, 0.000]\n",
      "Epoch 2 [184/340] - Loss: 56.028 [-55.876, 0.153, 0.000]\n",
      "Epoch 2 [185/340] - Loss: 41.243 [-41.105, 0.138, 0.000]\n",
      "Epoch 2 [186/340] - Loss: 45.017 [-44.886, 0.131, 0.000]\n",
      "Epoch 2 [187/340] - Loss: 58.644 [-58.510, 0.134, 0.000]\n",
      "Epoch 2 [188/340] - Loss: 54.658 [-54.531, 0.127, 0.000]\n",
      "Epoch 2 [189/340] - Loss: 60.147 [-60.014, 0.133, 0.000]\n",
      "Epoch 2 [190/340] - Loss: 60.936 [-60.789, 0.146, 0.000]\n",
      "Epoch 2 [191/340] - Loss: 52.671 [-52.528, 0.143, 0.000]\n",
      "Epoch 2 [192/340] - Loss: 55.867 [-55.714, 0.153, 0.000]\n",
      "Epoch 2 [193/340] - Loss: 50.993 [-50.826, 0.167, 0.000]\n",
      "Epoch 2 [194/340] - Loss: 54.340 [-54.151, 0.189, 0.000]\n",
      "Epoch 2 [195/340] - Loss: 58.775 [-58.559, 0.216, 0.000]\n",
      "Epoch 2 [196/340] - Loss: 51.520 [-51.286, 0.234, 0.000]\n",
      "Epoch 2 [197/340] - Loss: 51.410 [-51.151, 0.259, 0.000]\n",
      "Epoch 2 [198/340] - Loss: 57.978 [-57.680, 0.298, 0.000]\n",
      "Epoch 2 [199/340] - Loss: 55.185 [-54.882, 0.302, 0.000]\n",
      "Epoch 2 [200/340] - Loss: 53.687 [-53.403, 0.284, 0.000]\n",
      "Epoch 2 [201/340] - Loss: 54.693 [-54.468, 0.225, 0.000]\n",
      "Epoch 2 [202/340] - Loss: 54.085 [-53.918, 0.166, 0.000]\n",
      "Epoch 2 [203/340] - Loss: 50.949 [-50.827, 0.122, 0.000]\n",
      "Epoch 2 [204/340] - Loss: 74.389 [-74.284, 0.106, 0.000]\n",
      "Epoch 2 [205/340] - Loss: 55.169 [-55.086, 0.083, 0.000]\n",
      "Epoch 2 [206/340] - Loss: 57.672 [-57.597, 0.074, 0.000]\n",
      "Epoch 2 [207/340] - Loss: 49.525 [-49.455, 0.070, 0.000]\n",
      "Epoch 2 [208/340] - Loss: 49.327 [-49.257, 0.070, 0.000]\n",
      "Epoch 2 [209/340] - Loss: 53.787 [-53.715, 0.072, 0.000]\n",
      "Epoch 2 [210/340] - Loss: 51.743 [-51.666, 0.078, 0.000]\n",
      "Epoch 2 [211/340] - Loss: 52.738 [-52.651, 0.086, 0.000]\n",
      "Epoch 2 [212/340] - Loss: 51.721 [-51.624, 0.097, 0.000]\n",
      "Epoch 2 [213/340] - Loss: 52.948 [-52.840, 0.108, 0.000]\n",
      "Epoch 2 [214/340] - Loss: 50.572 [-50.453, 0.118, 0.000]\n",
      "Epoch 2 [215/340] - Loss: 50.395 [-50.270, 0.125, 0.000]\n",
      "Epoch 2 [216/340] - Loss: 44.315 [-44.187, 0.128, 0.000]\n",
      "Epoch 2 [217/340] - Loss: 53.064 [-52.933, 0.130, 0.000]\n",
      "Epoch 2 [218/340] - Loss: 51.786 [-51.659, 0.128, 0.000]\n",
      "Epoch 2 [219/340] - Loss: 44.016 [-43.889, 0.127, 0.000]\n",
      "Epoch 2 [220/340] - Loss: 45.492 [-45.369, 0.124, 0.000]\n",
      "Epoch 2 [221/340] - Loss: 48.087 [-47.966, 0.121, 0.000]\n",
      "Epoch 2 [222/340] - Loss: 55.459 [-55.339, 0.120, 0.000]\n",
      "Epoch 2 [223/340] - Loss: 52.469 [-52.350, 0.119, 0.000]\n",
      "Epoch 2 [224/340] - Loss: 47.554 [-47.436, 0.118, 0.000]\n",
      "Epoch 2 [225/340] - Loss: 54.923 [-54.799, 0.124, 0.000]\n",
      "Epoch 2 [226/340] - Loss: 46.696 [-46.567, 0.129, 0.000]\n",
      "Epoch 2 [227/340] - Loss: 48.101 [-47.965, 0.136, 0.000]\n",
      "Epoch 2 [228/340] - Loss: 39.632 [-39.492, 0.140, 0.000]\n",
      "Epoch 2 [229/340] - Loss: 47.226 [-47.085, 0.141, 0.000]\n",
      "Epoch 2 [230/340] - Loss: 58.955 [-58.812, 0.143, 0.000]\n",
      "Epoch 2 [231/340] - Loss: 51.282 [-51.139, 0.143, 0.000]\n",
      "Epoch 2 [232/340] - Loss: 42.796 [-42.645, 0.151, 0.000]\n",
      "Epoch 2 [233/340] - Loss: 50.661 [-50.508, 0.153, 0.000]\n",
      "Epoch 2 [234/340] - Loss: 44.770 [-44.611, 0.159, 0.000]\n",
      "Epoch 2 [235/340] - Loss: 44.608 [-44.439, 0.169, 0.000]\n",
      "Epoch 2 [236/340] - Loss: 55.974 [-55.793, 0.181, 0.000]\n",
      "Epoch 2 [237/340] - Loss: 48.805 [-48.611, 0.194, 0.000]\n",
      "Epoch 2 [238/340] - Loss: 54.141 [-53.936, 0.205, 0.000]\n",
      "Epoch 2 [239/340] - Loss: 60.227 [-60.036, 0.191, 0.000]\n",
      "Epoch 2 [240/340] - Loss: 46.120 [-45.953, 0.167, 0.000]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 2 [241/340] - Loss: 48.736 [-48.579, 0.158, 0.000]\n",
      "Epoch 2 [242/340] - Loss: 57.485 [-57.325, 0.160, 0.000]\n",
      "Epoch 2 [243/340] - Loss: 43.428 [-43.267, 0.161, 0.000]\n",
      "Epoch 2 [244/340] - Loss: 51.841 [-51.670, 0.170, 0.000]\n",
      "Epoch 2 [245/340] - Loss: 43.725 [-43.555, 0.169, 0.000]\n",
      "Epoch 2 [246/340] - Loss: 41.940 [-41.769, 0.171, 0.000]\n",
      "Epoch 2 [247/340] - Loss: 46.186 [-46.004, 0.181, 0.000]\n",
      "Epoch 2 [248/340] - Loss: 54.817 [-54.634, 0.183, 0.000]\n",
      "Epoch 2 [249/340] - Loss: 46.823 [-46.627, 0.196, 0.000]\n",
      "Epoch 2 [250/340] - Loss: 51.243 [-51.027, 0.216, 0.000]\n",
      "Epoch 2 [251/340] - Loss: 68.936 [-68.693, 0.243, 0.000]\n",
      "Epoch 2 [252/340] - Loss: 50.416 [-50.189, 0.227, 0.000]\n",
      "Epoch 2 [253/340] - Loss: 53.493 [-53.264, 0.229, 0.000]\n",
      "Epoch 2 [254/340] - Loss: 48.587 [-48.347, 0.239, 0.000]\n",
      "Epoch 2 [255/340] - Loss: 53.814 [-53.553, 0.261, 0.000]\n",
      "Epoch 2 [256/340] - Loss: 58.104 [-57.813, 0.291, 0.000]\n",
      "Epoch 2 [257/340] - Loss: 52.860 [-52.559, 0.301, 0.000]\n",
      "Epoch 2 [258/340] - Loss: 67.286 [-66.972, 0.314, 0.000]\n",
      "Epoch 2 [259/340] - Loss: 76.303 [-76.012, 0.291, 0.000]\n",
      "Epoch 2 [260/340] - Loss: 59.712 [-59.445, 0.267, 0.000]\n",
      "Epoch 2 [261/340] - Loss: 52.555 [-52.315, 0.240, 0.000]\n",
      "Epoch 2 [262/340] - Loss: 53.047 [-52.826, 0.220, 0.000]\n",
      "Epoch 2 [263/340] - Loss: 55.210 [-55.008, 0.202, 0.000]\n",
      "Epoch 2 [264/340] - Loss: 51.290 [-51.104, 0.186, 0.000]\n",
      "Epoch 2 [265/340] - Loss: 53.835 [-53.664, 0.171, 0.000]\n",
      "Epoch 2 [266/340] - Loss: 55.060 [-54.902, 0.158, 0.000]\n",
      "Epoch 2 [267/340] - Loss: 69.211 [-69.057, 0.154, 0.000]\n",
      "Epoch 2 [268/340] - Loss: 51.687 [-51.549, 0.138, 0.000]\n",
      "Epoch 2 [269/340] - Loss: 55.295 [-55.164, 0.131, 0.000]\n",
      "Epoch 2 [270/340] - Loss: 55.536 [-55.406, 0.130, 0.000]\n",
      "Epoch 2 [271/340] - Loss: 51.008 [-50.881, 0.127, 0.000]\n",
      "Epoch 2 [272/340] - Loss: 55.185 [-55.071, 0.114, 0.000]\n",
      "Epoch 2 [273/340] - Loss: 53.226 [-53.116, 0.111, 0.000]\n",
      "Epoch 2 [274/340] - Loss: 54.729 [-54.619, 0.110, 0.000]\n",
      "Epoch 2 [275/340] - Loss: 58.798 [-58.685, 0.113, 0.000]\n",
      "Epoch 2 [276/340] - Loss: 50.688 [-50.578, 0.111, 0.000]\n",
      "Epoch 2 [277/340] - Loss: 55.969 [-55.848, 0.121, 0.000]\n",
      "Epoch 2 [278/340] - Loss: 45.612 [-45.492, 0.120, 0.000]\n",
      "Epoch 2 [279/340] - Loss: 50.355 [-50.231, 0.124, 0.000]\n",
      "Epoch 2 [280/340] - Loss: 51.918 [-51.784, 0.134, 0.000]\n",
      "Epoch 2 [281/340] - Loss: 51.252 [-51.118, 0.134, 0.000]\n",
      "Epoch 2 [282/340] - Loss: 50.541 [-50.408, 0.133, 0.000]\n",
      "Epoch 2 [283/340] - Loss: 53.735 [-53.602, 0.133, 0.000]\n",
      "Epoch 2 [284/340] - Loss: 43.913 [-43.783, 0.130, 0.000]\n",
      "Epoch 2 [285/340] - Loss: 49.370 [-49.238, 0.131, 0.000]\n",
      "Epoch 2 [286/340] - Loss: 46.129 [-45.992, 0.137, 0.000]\n",
      "Epoch 2 [287/340] - Loss: 42.039 [-41.892, 0.147, 0.000]\n",
      "Epoch 2 [288/340] - Loss: 47.124 [-46.965, 0.159, 0.000]\n",
      "Epoch 2 [289/340] - Loss: 46.114 [-45.941, 0.173, 0.000]\n",
      "Epoch 2 [290/340] - Loss: 43.722 [-43.533, 0.189, 0.000]\n",
      "Epoch 2 [291/340] - Loss: 53.530 [-53.314, 0.216, 0.000]\n",
      "Epoch 2 [292/340] - Loss: 44.794 [-44.577, 0.217, 0.000]\n",
      "Epoch 2 [293/340] - Loss: 47.364 [-47.149, 0.215, 0.000]\n",
      "Epoch 2 [294/340] - Loss: 47.054 [-46.842, 0.212, 0.000]\n",
      "Epoch 2 [295/340] - Loss: 50.063 [-49.863, 0.199, 0.000]\n",
      "Epoch 2 [296/340] - Loss: 45.904 [-45.719, 0.185, 0.000]\n",
      "Epoch 2 [297/340] - Loss: 44.853 [-44.679, 0.174, 0.000]\n",
      "Epoch 2 [298/340] - Loss: 43.336 [-43.168, 0.168, 0.000]\n",
      "Epoch 2 [299/340] - Loss: 46.303 [-46.136, 0.167, 0.000]\n",
      "Epoch 2 [300/340] - Loss: 42.286 [-42.121, 0.165, 0.000]\n",
      "Epoch 2 [301/340] - Loss: 37.725 [-37.557, 0.168, 0.000]\n",
      "Epoch 2 [302/340] - Loss: 46.572 [-46.398, 0.174, 0.000]\n",
      "Epoch 2 [303/340] - Loss: 42.903 [-42.717, 0.186, 0.000]\n",
      "Epoch 2 [304/340] - Loss: 43.273 [-43.071, 0.203, 0.000]\n",
      "Epoch 2 [305/340] - Loss: 44.829 [-44.610, 0.219, 0.000]\n",
      "Epoch 2 [306/340] - Loss: 45.242 [-45.014, 0.228, 0.000]\n",
      "Epoch 2 [307/340] - Loss: 40.886 [-40.661, 0.226, 0.000]\n",
      "Epoch 2 [308/340] - Loss: 42.246 [-42.032, 0.213, 0.000]\n",
      "Epoch 2 [309/340] - Loss: 42.926 [-42.731, 0.196, 0.000]\n",
      "Epoch 2 [310/340] - Loss: 44.190 [-44.010, 0.180, 0.000]\n",
      "Epoch 2 [311/340] - Loss: 39.620 [-39.455, 0.164, 0.000]\n",
      "Epoch 2 [312/340] - Loss: 42.296 [-42.140, 0.156, 0.000]\n",
      "Epoch 2 [313/340] - Loss: 45.856 [-45.705, 0.151, 0.000]\n",
      "Epoch 2 [314/340] - Loss: 41.726 [-41.574, 0.152, 0.000]\n",
      "Epoch 2 [315/340] - Loss: 39.246 [-39.092, 0.154, 0.000]\n",
      "Epoch 2 [316/340] - Loss: 55.055 [-54.882, 0.173, 0.000]\n",
      "Epoch 2 [317/340] - Loss: 54.824 [-54.641, 0.183, 0.000]\n",
      "Epoch 2 [318/340] - Loss: 40.815 [-40.611, 0.204, 0.000]\n",
      "Epoch 2 [319/340] - Loss: 45.939 [-45.712, 0.228, 0.000]\n",
      "Epoch 2 [320/340] - Loss: 54.747 [-54.513, 0.234, 0.000]\n",
      "Epoch 2 [321/340] - Loss: 41.164 [-40.952, 0.212, 0.000]\n",
      "Epoch 2 [322/340] - Loss: 42.227 [-42.028, 0.199, 0.000]\n",
      "Epoch 2 [323/340] - Loss: 42.228 [-42.037, 0.191, 0.000]\n",
      "Epoch 2 [324/340] - Loss: 43.965 [-43.772, 0.192, 0.000]\n",
      "Epoch 2 [325/340] - Loss: 44.004 [-43.799, 0.205, 0.000]\n",
      "Epoch 2 [326/340] - Loss: 47.493 [-47.272, 0.221, 0.000]\n",
      "Epoch 2 [327/340] - Loss: 44.528 [-44.292, 0.236, 0.000]\n",
      "Epoch 2 [328/340] - Loss: 40.171 [-39.929, 0.241, 0.000]\n",
      "Epoch 2 [329/340] - Loss: 42.875 [-42.674, 0.201, 0.000]\n",
      "Epoch 2 [330/340] - Loss: 45.731 [-45.554, 0.177, 0.000]\n",
      "Epoch 2 [331/340] - Loss: 45.968 [-45.807, 0.161, 0.000]\n",
      "Epoch 2 [332/340] - Loss: 40.797 [-40.649, 0.147, 0.000]\n",
      "Epoch 2 [333/340] - Loss: 39.997 [-39.855, 0.142, 0.000]\n",
      "Epoch 2 [334/340] - Loss: 39.575 [-39.436, 0.140, 0.000]\n",
      "Epoch 2 [335/340] - Loss: 42.963 [-42.819, 0.144, 0.000]\n",
      "Epoch 2 [336/340] - Loss: 43.390 [-43.250, 0.139, 0.000]\n",
      "Epoch 2 [337/340] - Loss: 39.960 [-39.816, 0.143, 0.000]\n",
      "Epoch 2 [338/340] - Loss: 39.083 [-38.931, 0.152, 0.000]\n",
      "Epoch 2 [339/340] - Loss: 38.110 [-37.954, 0.156, 0.000]\n",
      "Epoch 3 [0/340] - Loss: 39.153 [-38.992, 0.160, 0.000]\n",
      "Epoch 3 [1/340] - Loss: 39.925 [-39.765, 0.160, 0.000]\n",
      "Epoch 3 [2/340] - Loss: 40.506 [-40.346, 0.160, 0.000]\n",
      "Epoch 3 [3/340] - Loss: 37.606 [-37.449, 0.158, 0.000]\n",
      "Epoch 3 [4/340] - Loss: 40.057 [-39.898, 0.158, 0.000]\n",
      "Epoch 3 [5/340] - Loss: 34.232 [-34.070, 0.162, 0.000]\n",
      "Epoch 3 [6/340] - Loss: 38.473 [-38.302, 0.171, 0.000]\n",
      "Epoch 3 [7/340] - Loss: 36.647 [-36.465, 0.183, 0.000]\n",
      "Epoch 3 [8/340] - Loss: 45.657 [-45.457, 0.199, 0.000]\n",
      "Epoch 3 [9/340] - Loss: 39.810 [-39.600, 0.210, 0.000]\n",
      "Epoch 3 [10/340] - Loss: 38.516 [-38.286, 0.230, 0.000]\n",
      "Epoch 3 [11/340] - Loss: 38.695 [-38.451, 0.244, 0.000]\n",
      "Epoch 3 [12/340] - Loss: 37.532 [-37.281, 0.251, 0.000]\n",
      "Epoch 3 [13/340] - Loss: 38.194 [-37.939, 0.255, 0.000]\n",
      "Epoch 3 [14/340] - Loss: 37.260 [-37.004, 0.256, 0.000]\n",
      "Epoch 3 [15/340] - Loss: 45.987 [-45.735, 0.252, 0.000]\n",
      "Epoch 3 [16/340] - Loss: 35.529 [-35.295, 0.234, 0.000]\n",
      "Epoch 3 [17/340] - Loss: 46.720 [-46.494, 0.226, 0.000]\n",
      "Epoch 3 [18/340] - Loss: 37.481 [-37.278, 0.203, 0.000]\n",
      "Epoch 3 [19/340] - Loss: 39.089 [-38.899, 0.190, 0.000]\n",
      "Epoch 3 [20/340] - Loss: 44.761 [-44.586, 0.175, 0.000]\n",
      "Epoch 3 [21/340] - Loss: 39.575 [-39.412, 0.163, 0.000]\n",
      "Epoch 3 [22/340] - Loss: 41.394 [-41.237, 0.157, 0.000]\n",
      "Epoch 3 [23/340] - Loss: 40.211 [-40.055, 0.156, 0.000]\n",
      "Epoch 3 [24/340] - Loss: 34.067 [-33.914, 0.153, 0.000]\n",
      "Epoch 3 [25/340] - Loss: 37.119 [-36.967, 0.151, 0.000]\n",
      "Epoch 3 [26/340] - Loss: 34.336 [-34.187, 0.149, 0.000]\n",
      "Epoch 3 [27/340] - Loss: 38.486 [-38.339, 0.147, 0.000]\n",
      "Epoch 3 [28/340] - Loss: 39.279 [-39.135, 0.144, 0.000]\n",
      "Epoch 3 [29/340] - Loss: 39.746 [-39.605, 0.141, 0.000]\n",
      "Epoch 3 [30/340] - Loss: 37.893 [-37.751, 0.142, 0.000]\n",
      "Epoch 3 [31/340] - Loss: 35.036 [-34.892, 0.143, 0.000]\n",
      "Epoch 3 [32/340] - Loss: 41.463 [-41.311, 0.152, 0.000]\n",
      "Epoch 3 [33/340] - Loss: 35.876 [-35.711, 0.164, 0.000]\n",
      "Epoch 3 [34/340] - Loss: 32.589 [-32.406, 0.183, 0.000]\n",
      "Epoch 3 [35/340] - Loss: 40.316 [-40.109, 0.207, 0.000]\n",
      "Epoch 3 [36/340] - Loss: 34.204 [-33.968, 0.235, 0.000]\n",
      "Epoch 3 [37/340] - Loss: 39.548 [-39.278, 0.269, 0.000]\n",
      "Epoch 3 [38/340] - Loss: 32.880 [-32.556, 0.324, 0.000]\n",
      "Epoch 3 [39/340] - Loss: 39.125 [-38.755, 0.370, 0.000]\n",
      "Epoch 3 [40/340] - Loss: 38.131 [-37.769, 0.363, 0.000]\n",
      "Epoch 3 [41/340] - Loss: 40.258 [-39.922, 0.337, 0.000]\n",
      "Epoch 3 [42/340] - Loss: 44.585 [-44.277, 0.308, 0.000]\n",
      "Epoch 3 [43/340] - Loss: 42.699 [-42.429, 0.270, 0.000]\n",
      "Epoch 3 [44/340] - Loss: 41.286 [-41.039, 0.247, 0.000]\n",
      "Epoch 3 [45/340] - Loss: 38.416 [-38.185, 0.231, 0.000]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 3 [46/340] - Loss: 51.017 [-50.779, 0.238, 0.000]\n",
      "Epoch 3 [47/340] - Loss: 37.612 [-37.402, 0.210, 0.000]\n",
      "Epoch 3 [48/340] - Loss: 39.810 [-39.617, 0.193, 0.000]\n",
      "Epoch 3 [49/340] - Loss: 38.108 [-37.911, 0.196, 0.000]\n",
      "Epoch 3 [50/340] - Loss: 40.810 [-40.621, 0.189, 0.000]\n",
      "Epoch 3 [51/340] - Loss: 35.829 [-35.639, 0.190, 0.000]\n",
      "Epoch 3 [52/340] - Loss: 39.424 [-39.230, 0.194, 0.000]\n",
      "Epoch 3 [53/340] - Loss: 40.097 [-39.902, 0.195, 0.000]\n",
      "Epoch 3 [54/340] - Loss: 46.032 [-45.834, 0.199, 0.000]\n",
      "Epoch 3 [55/340] - Loss: 38.829 [-38.641, 0.188, 0.000]\n",
      "Epoch 3 [56/340] - Loss: 38.512 [-38.325, 0.186, 0.000]\n",
      "Epoch 3 [57/340] - Loss: 37.688 [-37.516, 0.171, 0.000]\n",
      "Epoch 3 [58/340] - Loss: 38.388 [-38.220, 0.169, 0.000]\n",
      "Epoch 3 [59/340] - Loss: 39.455 [-39.297, 0.158, 0.000]\n",
      "Epoch 3 [60/340] - Loss: 41.008 [-40.850, 0.158, 0.000]\n",
      "Epoch 3 [61/340] - Loss: 34.305 [-34.146, 0.159, 0.000]\n",
      "Epoch 3 [62/340] - Loss: 37.840 [-37.674, 0.166, 0.000]\n",
      "Epoch 3 [63/340] - Loss: 36.268 [-36.096, 0.172, 0.000]\n",
      "Epoch 3 [64/340] - Loss: 33.772 [-33.583, 0.190, 0.000]\n",
      "Epoch 3 [65/340] - Loss: 40.659 [-40.455, 0.204, 0.000]\n",
      "Epoch 3 [66/340] - Loss: 38.233 [-38.022, 0.211, 0.000]\n",
      "Epoch 3 [67/340] - Loss: 38.971 [-38.751, 0.220, 0.000]\n",
      "Epoch 3 [68/340] - Loss: 40.220 [-39.999, 0.220, 0.000]\n",
      "Epoch 3 [69/340] - Loss: 37.332 [-37.110, 0.222, 0.000]\n",
      "Epoch 3 [70/340] - Loss: 34.837 [-34.602, 0.236, 0.000]\n",
      "Epoch 3 [71/340] - Loss: 34.465 [-34.220, 0.245, 0.000]\n",
      "Epoch 3 [72/340] - Loss: 38.245 [-37.982, 0.263, 0.000]\n",
      "Epoch 3 [73/340] - Loss: 37.877 [-37.581, 0.296, 0.000]\n",
      "Epoch 3 [74/340] - Loss: 40.750 [-40.448, 0.303, 0.000]\n",
      "Epoch 3 [75/340] - Loss: 38.794 [-38.499, 0.295, 0.000]\n",
      "Epoch 3 [76/340] - Loss: 36.969 [-36.696, 0.273, 0.000]\n",
      "Epoch 3 [77/340] - Loss: 36.544 [-36.312, 0.233, 0.000]\n",
      "Epoch 3 [78/340] - Loss: 40.494 [-40.289, 0.205, 0.000]\n",
      "Epoch 3 [79/340] - Loss: 40.058 [-39.869, 0.189, 0.000]\n",
      "Epoch 3 [80/340] - Loss: 35.630 [-35.460, 0.170, 0.000]\n",
      "Epoch 3 [81/340] - Loss: 36.126 [-35.965, 0.161, 0.000]\n",
      "Epoch 3 [82/340] - Loss: 34.082 [-33.922, 0.160, 0.000]\n",
      "Epoch 3 [83/340] - Loss: 34.331 [-34.171, 0.160, 0.000]\n",
      "Epoch 3 [84/340] - Loss: 37.338 [-37.174, 0.164, 0.000]\n",
      "Epoch 3 [85/340] - Loss: 38.489 [-38.313, 0.176, 0.000]\n",
      "Epoch 3 [86/340] - Loss: 35.793 [-35.603, 0.189, 0.000]\n",
      "Epoch 3 [87/340] - Loss: 34.680 [-34.484, 0.196, 0.000]\n",
      "Epoch 3 [88/340] - Loss: 29.417 [-29.219, 0.198, 0.000]\n",
      "Epoch 3 [89/340] - Loss: 35.761 [-35.564, 0.197, 0.000]\n",
      "Epoch 3 [90/340] - Loss: 32.931 [-32.736, 0.195, 0.000]\n",
      "Epoch 3 [91/340] - Loss: 33.567 [-33.373, 0.194, 0.000]\n",
      "Epoch 3 [92/340] - Loss: 37.335 [-37.136, 0.199, 0.000]\n",
      "Epoch 3 [93/340] - Loss: 36.165 [-35.967, 0.198, 0.000]\n",
      "Epoch 3 [94/340] - Loss: 31.511 [-31.303, 0.208, 0.000]\n",
      "Epoch 3 [95/340] - Loss: 36.934 [-36.725, 0.209, 0.000]\n",
      "Epoch 3 [96/340] - Loss: 36.094 [-35.880, 0.214, 0.000]\n",
      "Epoch 3 [97/340] - Loss: 36.558 [-36.342, 0.216, 0.000]\n",
      "Epoch 3 [98/340] - Loss: 36.973 [-36.754, 0.219, 0.000]\n",
      "Epoch 3 [99/340] - Loss: 34.887 [-34.667, 0.220, 0.000]\n",
      "Epoch 3 [100/340] - Loss: 35.543 [-35.320, 0.223, 0.000]\n",
      "Epoch 3 [101/340] - Loss: 36.205 [-35.972, 0.233, 0.000]\n",
      "Epoch 3 [102/340] - Loss: 38.408 [-38.177, 0.231, 0.000]\n",
      "Epoch 3 [103/340] - Loss: 36.681 [-36.435, 0.245, 0.000]\n",
      "Epoch 3 [104/340] - Loss: 36.028 [-35.790, 0.238, 0.000]\n",
      "Epoch 3 [105/340] - Loss: 36.988 [-36.753, 0.235, 0.000]\n",
      "Epoch 3 [106/340] - Loss: 31.903 [-31.672, 0.231, 0.000]\n",
      "Epoch 3 [107/340] - Loss: 37.494 [-37.267, 0.227, 0.000]\n",
      "Epoch 3 [108/340] - Loss: 35.290 [-35.070, 0.220, 0.000]\n",
      "Epoch 3 [109/340] - Loss: 33.309 [-33.091, 0.218, 0.000]\n",
      "Epoch 3 [110/340] - Loss: 33.097 [-32.885, 0.213, 0.000]\n",
      "Epoch 3 [111/340] - Loss: 37.135 [-36.926, 0.208, 0.000]\n",
      "Epoch 3 [112/340] - Loss: 32.518 [-32.307, 0.211, 0.000]\n",
      "Epoch 3 [113/340] - Loss: 34.642 [-34.424, 0.217, 0.000]\n",
      "Epoch 3 [114/340] - Loss: 35.390 [-35.163, 0.227, 0.000]\n",
      "Epoch 3 [115/340] - Loss: 36.068 [-35.824, 0.244, 0.000]\n",
      "Epoch 3 [116/340] - Loss: 33.709 [-33.455, 0.254, 0.000]\n",
      "Epoch 3 [117/340] - Loss: 35.983 [-35.716, 0.267, 0.000]\n",
      "Epoch 3 [118/340] - Loss: 32.898 [-32.612, 0.286, 0.000]\n",
      "Epoch 3 [119/340] - Loss: 35.328 [-35.027, 0.301, 0.000]\n",
      "Epoch 3 [120/340] - Loss: 36.018 [-35.717, 0.302, 0.000]\n",
      "Epoch 3 [121/340] - Loss: 34.850 [-34.594, 0.256, 0.000]\n",
      "Epoch 3 [122/340] - Loss: 38.657 [-38.418, 0.239, 0.000]\n",
      "Epoch 3 [123/340] - Loss: 36.547 [-36.325, 0.222, 0.000]\n",
      "Epoch 3 [124/340] - Loss: 33.667 [-33.466, 0.201, 0.000]\n",
      "Epoch 3 [125/340] - Loss: 29.827 [-29.633, 0.194, 0.000]\n",
      "Epoch 3 [126/340] - Loss: 38.013 [-37.809, 0.204, 0.000]\n",
      "Epoch 3 [127/340] - Loss: 33.898 [-33.695, 0.204, 0.000]\n",
      "Epoch 3 [128/340] - Loss: 31.717 [-31.508, 0.209, 0.000]\n",
      "Epoch 3 [129/340] - Loss: 34.017 [-33.798, 0.219, 0.000]\n",
      "Epoch 3 [130/340] - Loss: 32.560 [-32.333, 0.227, 0.000]\n",
      "Epoch 3 [131/340] - Loss: 36.334 [-36.104, 0.230, 0.000]\n",
      "Epoch 3 [132/340] - Loss: 34.821 [-34.590, 0.231, 0.000]\n",
      "Epoch 3 [133/340] - Loss: 34.366 [-34.136, 0.230, 0.000]\n",
      "Epoch 3 [134/340] - Loss: 33.629 [-33.404, 0.226, 0.000]\n",
      "Epoch 3 [135/340] - Loss: 32.898 [-32.668, 0.230, 0.000]\n",
      "Epoch 3 [136/340] - Loss: 31.503 [-31.278, 0.225, 0.000]\n",
      "Epoch 3 [137/340] - Loss: 30.959 [-30.727, 0.232, 0.000]\n",
      "Epoch 3 [138/340] - Loss: 34.954 [-34.706, 0.248, 0.000]\n",
      "Epoch 3 [139/340] - Loss: 35.472 [-35.206, 0.266, 0.000]\n",
      "Epoch 3 [140/340] - Loss: 34.013 [-33.741, 0.272, 0.000]\n",
      "Epoch 3 [141/340] - Loss: 36.895 [-36.609, 0.285, 0.000]\n",
      "Epoch 3 [142/340] - Loss: 38.428 [-38.120, 0.308, 0.000]\n",
      "Epoch 3 [143/340] - Loss: 35.833 [-35.543, 0.290, 0.000]\n",
      "Epoch 3 [144/340] - Loss: 31.196 [-30.902, 0.294, 0.000]\n",
      "Epoch 3 [145/340] - Loss: 39.527 [-39.208, 0.319, 0.000]\n",
      "Epoch 3 [146/340] - Loss: 34.225 [-33.907, 0.317, 0.000]\n",
      "Epoch 3 [147/340] - Loss: 32.911 [-32.583, 0.329, 0.000]\n",
      "Epoch 3 [148/340] - Loss: 37.989 [-37.628, 0.361, 0.000]\n",
      "Epoch 3 [149/340] - Loss: 33.009 [-32.648, 0.361, 0.000]\n",
      "Epoch 3 [150/340] - Loss: 38.481 [-38.116, 0.366, 0.000]\n",
      "Epoch 3 [151/340] - Loss: 32.567 [-32.195, 0.372, 0.000]\n",
      "Epoch 3 [152/340] - Loss: 32.725 [-32.362, 0.363, 0.000]\n",
      "Epoch 3 [153/340] - Loss: 34.156 [-33.804, 0.352, 0.000]\n",
      "Epoch 3 [154/340] - Loss: 36.963 [-36.596, 0.367, 0.000]\n",
      "Epoch 3 [155/340] - Loss: 38.848 [-38.514, 0.334, 0.000]\n",
      "Epoch 3 [156/340] - Loss: 30.219 [-29.885, 0.334, 0.000]\n",
      "Epoch 3 [157/340] - Loss: 42.680 [-42.317, 0.362, 0.000]\n",
      "Epoch 3 [158/340] - Loss: 31.901 [-31.564, 0.337, 0.000]\n",
      "Epoch 3 [159/340] - Loss: 31.423 [-31.095, 0.328, 0.000]\n",
      "Epoch 3 [160/340] - Loss: 39.328 [-38.995, 0.333, 0.000]\n",
      "Epoch 3 [161/340] - Loss: 39.222 [-38.905, 0.317, 0.000]\n",
      "Epoch 3 [162/340] - Loss: 36.275 [-35.984, 0.291, 0.000]\n",
      "Epoch 3 [163/340] - Loss: 35.758 [-35.478, 0.280, 0.000]\n",
      "Epoch 3 [164/340] - Loss: 35.412 [-35.138, 0.274, 0.000]\n",
      "Epoch 3 [165/340] - Loss: 36.858 [-36.571, 0.287, 0.000]\n",
      "Epoch 3 [166/340] - Loss: 36.717 [-36.435, 0.283, 0.000]\n",
      "Epoch 3 [167/340] - Loss: 33.852 [-33.568, 0.284, 0.000]\n",
      "Epoch 3 [168/340] - Loss: 39.190 [-38.895, 0.296, 0.000]\n",
      "Epoch 3 [169/340] - Loss: 37.360 [-37.073, 0.288, 0.000]\n",
      "Epoch 3 [170/340] - Loss: 37.747 [-37.458, 0.288, 0.000]\n",
      "Epoch 3 [171/340] - Loss: 36.122 [-35.825, 0.297, 0.000]\n",
      "Epoch 3 [172/340] - Loss: 33.484 [-33.189, 0.295, 0.000]\n",
      "Epoch 3 [173/340] - Loss: 31.219 [-30.926, 0.293, 0.000]\n",
      "Epoch 3 [174/340] - Loss: 34.861 [-34.569, 0.292, 0.000]\n",
      "Epoch 3 [175/340] - Loss: 37.874 [-37.582, 0.293, 0.000]\n",
      "Epoch 3 [176/340] - Loss: 33.600 [-33.298, 0.302, 0.000]\n",
      "Epoch 3 [177/340] - Loss: 34.649 [-34.343, 0.306, 0.000]\n",
      "Epoch 3 [178/340] - Loss: 35.109 [-34.794, 0.316, 0.000]\n",
      "Epoch 3 [179/340] - Loss: 33.866 [-33.547, 0.319, 0.000]\n",
      "Epoch 3 [180/340] - Loss: 33.641 [-33.315, 0.326, 0.000]\n",
      "Epoch 3 [181/340] - Loss: 36.248 [-35.895, 0.352, 0.000]\n",
      "Epoch 3 [182/340] - Loss: 35.625 [-35.269, 0.356, 0.000]\n",
      "Epoch 3 [183/340] - Loss: 33.182 [-32.823, 0.359, 0.000]\n",
      "Epoch 3 [184/340] - Loss: 30.427 [-30.061, 0.366, 0.000]\n",
      "Epoch 3 [185/340] - Loss: 33.195 [-32.803, 0.392, 0.000]\n",
      "Epoch 3 [186/340] - Loss: 35.028 [-34.664, 0.364, 0.000]\n",
      "Epoch 3 [187/340] - Loss: 34.377 [-34.013, 0.364, 0.000]\n",
      "Epoch 3 [188/340] - Loss: 33.042 [-32.665, 0.377, 0.000]\n",
      "Epoch 3 [189/340] - Loss: 34.681 [-34.325, 0.356, 0.000]\n",
      "Epoch 3 [190/340] - Loss: 31.748 [-31.422, 0.326, 0.000]\n",
      "Epoch 3 [191/340] - Loss: 35.566 [-35.251, 0.315, 0.000]\n",
      "Epoch 3 [192/340] - Loss: 33.970 [-33.667, 0.303, 0.000]\n",
      "Epoch 3 [193/340] - Loss: 34.725 [-34.426, 0.298, 0.000]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 3 [194/340] - Loss: 33.477 [-33.197, 0.280, 0.000]\n",
      "Epoch 3 [195/340] - Loss: 30.964 [-30.692, 0.271, 0.000]\n",
      "Epoch 3 [196/340] - Loss: 34.134 [-33.877, 0.257, 0.000]\n",
      "Epoch 3 [197/340] - Loss: 32.934 [-32.677, 0.257, 0.000]\n",
      "Epoch 3 [198/340] - Loss: 37.583 [-37.331, 0.251, 0.000]\n",
      "Epoch 3 [199/340] - Loss: 33.555 [-33.299, 0.256, 0.000]\n",
      "Epoch 3 [200/340] - Loss: 33.746 [-33.490, 0.256, 0.000]\n",
      "Epoch 3 [201/340] - Loss: 35.094 [-34.844, 0.250, 0.000]\n",
      "Epoch 3 [202/340] - Loss: 32.439 [-32.182, 0.256, 0.000]\n",
      "Epoch 3 [203/340] - Loss: 33.123 [-32.864, 0.259, 0.000]\n",
      "Epoch 3 [204/340] - Loss: 33.461 [-33.200, 0.260, 0.000]\n",
      "Epoch 3 [205/340] - Loss: 33.694 [-33.429, 0.264, 0.000]\n",
      "Epoch 3 [206/340] - Loss: 34.111 [-33.817, 0.294, 0.000]\n",
      "Epoch 3 [207/340] - Loss: 34.630 [-34.311, 0.319, 0.000]\n",
      "Epoch 3 [208/340] - Loss: 34.324 [-33.987, 0.337, 0.000]\n",
      "Epoch 3 [209/340] - Loss: 33.232 [-32.880, 0.352, 0.000]\n",
      "Epoch 3 [210/340] - Loss: 33.556 [-33.201, 0.355, 0.000]\n",
      "Epoch 3 [211/340] - Loss: 32.138 [-31.767, 0.371, 0.000]\n",
      "Epoch 3 [212/340] - Loss: 35.105 [-34.709, 0.395, 0.000]\n",
      "Epoch 3 [213/340] - Loss: 32.906 [-32.493, 0.413, 0.000]\n",
      "Epoch 3 [214/340] - Loss: 33.054 [-32.667, 0.387, 0.000]\n",
      "Epoch 3 [215/340] - Loss: 31.627 [-31.266, 0.362, 0.000]\n",
      "Epoch 3 [216/340] - Loss: 32.834 [-32.480, 0.354, 0.000]\n",
      "Epoch 3 [217/340] - Loss: 33.087 [-32.750, 0.337, 0.000]\n",
      "Epoch 3 [218/340] - Loss: 38.352 [-38.014, 0.339, 0.000]\n",
      "Epoch 3 [219/340] - Loss: 32.321 [-31.988, 0.333, 0.000]\n",
      "Epoch 3 [220/340] - Loss: 33.751 [-33.430, 0.321, 0.000]\n",
      "Epoch 3 [221/340] - Loss: 32.623 [-32.314, 0.308, 0.000]\n",
      "Epoch 3 [222/340] - Loss: 35.755 [-35.453, 0.302, 0.000]\n",
      "Epoch 3 [223/340] - Loss: 32.764 [-32.469, 0.295, 0.000]\n",
      "Epoch 3 [224/340] - Loss: 37.423 [-37.125, 0.298, 0.000]\n",
      "Epoch 3 [225/340] - Loss: 37.151 [-36.836, 0.315, 0.000]\n",
      "Epoch 3 [226/340] - Loss: 35.006 [-34.699, 0.307, 0.000]\n",
      "Epoch 3 [227/340] - Loss: 33.729 [-33.411, 0.317, 0.000]\n",
      "Epoch 3 [228/340] - Loss: 34.415 [-34.088, 0.328, 0.000]\n",
      "Epoch 3 [229/340] - Loss: 34.135 [-33.813, 0.322, 0.000]\n",
      "Epoch 3 [230/340] - Loss: 35.211 [-34.905, 0.306, 0.000]\n",
      "Epoch 3 [231/340] - Loss: 33.006 [-32.728, 0.278, 0.000]\n",
      "Epoch 3 [232/340] - Loss: 34.812 [-34.546, 0.266, 0.000]\n",
      "Epoch 3 [233/340] - Loss: 32.752 [-32.506, 0.245, 0.000]\n",
      "Epoch 3 [234/340] - Loss: 34.523 [-34.282, 0.240, 0.000]\n",
      "Epoch 3 [235/340] - Loss: 32.336 [-32.095, 0.241, 0.000]\n",
      "Epoch 3 [236/340] - Loss: 33.927 [-33.674, 0.254, 0.000]\n",
      "Epoch 3 [237/340] - Loss: 35.838 [-35.569, 0.268, 0.000]\n",
      "Epoch 3 [238/340] - Loss: 32.286 [-32.008, 0.278, 0.000]\n",
      "Epoch 3 [239/340] - Loss: 30.115 [-29.854, 0.261, 0.000]\n",
      "Epoch 3 [240/340] - Loss: 37.495 [-37.236, 0.259, 0.000]\n",
      "Epoch 3 [241/340] - Loss: 31.659 [-31.408, 0.251, 0.000]\n",
      "Epoch 3 [242/340] - Loss: 30.832 [-30.571, 0.261, 0.000]\n",
      "Epoch 3 [243/340] - Loss: 33.006 [-32.744, 0.262, 0.000]\n",
      "Epoch 3 [244/340] - Loss: 33.723 [-33.460, 0.263, 0.000]\n",
      "Epoch 3 [245/340] - Loss: 34.879 [-34.595, 0.283, 0.000]\n",
      "Epoch 3 [246/340] - Loss: 33.247 [-32.942, 0.305, 0.000]\n",
      "Epoch 3 [247/340] - Loss: 34.071 [-33.750, 0.321, 0.000]\n",
      "Epoch 3 [248/340] - Loss: 33.772 [-33.433, 0.338, 0.000]\n",
      "Epoch 3 [249/340] - Loss: 32.452 [-32.110, 0.342, 0.000]\n",
      "Epoch 3 [250/340] - Loss: 29.744 [-29.407, 0.337, 0.000]\n",
      "Epoch 3 [251/340] - Loss: 31.795 [-31.455, 0.340, 0.000]\n",
      "Epoch 3 [252/340] - Loss: 29.080 [-28.749, 0.331, 0.000]\n",
      "Epoch 3 [253/340] - Loss: 32.965 [-32.623, 0.341, 0.000]\n",
      "Epoch 3 [254/340] - Loss: 31.180 [-30.831, 0.349, 0.000]\n",
      "Epoch 3 [255/340] - Loss: 31.346 [-30.983, 0.363, 0.000]\n",
      "Epoch 3 [256/340] - Loss: 31.974 [-31.601, 0.373, 0.000]\n",
      "Epoch 3 [257/340] - Loss: 33.243 [-32.859, 0.384, 0.000]\n",
      "Epoch 3 [258/340] - Loss: 28.911 [-28.532, 0.380, 0.000]\n",
      "Epoch 3 [259/340] - Loss: 33.597 [-33.226, 0.370, 0.000]\n",
      "Epoch 3 [260/340] - Loss: 30.485 [-30.112, 0.373, 0.000]\n",
      "Epoch 3 [261/340] - Loss: 33.305 [-32.939, 0.366, 0.000]\n",
      "Epoch 3 [262/340] - Loss: 28.671 [-28.306, 0.364, 0.000]\n",
      "Epoch 3 [263/340] - Loss: 30.041 [-29.660, 0.381, 0.000]\n",
      "Epoch 3 [264/340] - Loss: 29.085 [-28.714, 0.371, 0.000]\n",
      "Epoch 3 [265/340] - Loss: 32.350 [-31.980, 0.370, 0.000]\n",
      "Epoch 3 [266/340] - Loss: 32.541 [-32.152, 0.389, 0.000]\n",
      "Epoch 3 [267/340] - Loss: 30.685 [-30.307, 0.379, 0.000]\n",
      "Epoch 3 [268/340] - Loss: 30.002 [-29.606, 0.396, 0.000]\n",
      "Epoch 3 [269/340] - Loss: 30.697 [-30.301, 0.395, 0.000]\n",
      "Epoch 3 [270/340] - Loss: 33.814 [-33.422, 0.392, 0.000]\n",
      "Epoch 3 [271/340] - Loss: 33.131 [-32.753, 0.378, 0.000]\n",
      "Epoch 3 [272/340] - Loss: 27.583 [-27.205, 0.378, 0.000]\n",
      "Epoch 3 [273/340] - Loss: 29.680 [-29.300, 0.381, 0.000]\n",
      "Epoch 3 [274/340] - Loss: 30.791 [-30.397, 0.393, 0.000]\n",
      "Epoch 3 [275/340] - Loss: 27.594 [-27.205, 0.389, 0.000]\n",
      "Epoch 3 [276/340] - Loss: 27.001 [-26.596, 0.405, 0.000]\n",
      "Epoch 3 [277/340] - Loss: 30.343 [-29.937, 0.406, 0.000]\n",
      "Epoch 3 [278/340] - Loss: 27.328 [-26.944, 0.385, 0.000]\n",
      "Epoch 3 [279/340] - Loss: 29.518 [-29.132, 0.386, 0.000]\n",
      "Epoch 3 [280/340] - Loss: 26.428 [-26.036, 0.392, 0.000]\n",
      "Epoch 3 [281/340] - Loss: 30.228 [-29.841, 0.387, 0.000]\n",
      "Epoch 3 [282/340] - Loss: 31.369 [-30.958, 0.411, 0.000]\n",
      "Epoch 3 [283/340] - Loss: 32.008 [-31.562, 0.446, 0.000]\n",
      "Epoch 3 [284/340] - Loss: 27.779 [-27.320, 0.458, 0.000]\n",
      "Epoch 3 [285/340] - Loss: 26.576 [-26.108, 0.468, 0.000]\n",
      "Epoch 3 [286/340] - Loss: 29.550 [-29.109, 0.441, 0.000]\n",
      "Epoch 3 [287/340] - Loss: 28.201 [-27.800, 0.401, 0.000]\n",
      "Epoch 3 [288/340] - Loss: 26.975 [-26.601, 0.374, 0.000]\n",
      "Epoch 3 [289/340] - Loss: 30.275 [-29.924, 0.351, 0.000]\n",
      "Epoch 3 [290/340] - Loss: 30.521 [-30.188, 0.333, 0.000]\n",
      "Epoch 3 [291/340] - Loss: 27.617 [-27.291, 0.326, 0.000]\n",
      "Epoch 3 [292/340] - Loss: 30.275 [-29.952, 0.323, 0.000]\n",
      "Epoch 3 [293/340] - Loss: 29.759 [-29.430, 0.330, 0.000]\n",
      "Epoch 3 [294/340] - Loss: 28.802 [-28.456, 0.345, 0.000]\n",
      "Epoch 3 [295/340] - Loss: 29.776 [-29.409, 0.367, 0.000]\n",
      "Epoch 3 [296/340] - Loss: 27.723 [-27.333, 0.391, 0.000]\n",
      "Epoch 3 [297/340] - Loss: 33.297 [-32.898, 0.400, 0.000]\n",
      "Epoch 3 [298/340] - Loss: 27.866 [-27.466, 0.400, 0.000]\n",
      "Epoch 3 [299/340] - Loss: 30.848 [-30.450, 0.399, 0.000]\n",
      "Epoch 3 [300/340] - Loss: 29.307 [-28.915, 0.391, 0.000]\n",
      "Epoch 3 [301/340] - Loss: 28.148 [-27.768, 0.380, 0.000]\n",
      "Epoch 3 [302/340] - Loss: 30.744 [-30.376, 0.367, 0.000]\n",
      "Epoch 3 [303/340] - Loss: 28.564 [-28.208, 0.357, 0.000]\n",
      "Epoch 3 [304/340] - Loss: 29.559 [-29.207, 0.352, 0.000]\n",
      "Epoch 3 [305/340] - Loss: 27.222 [-26.869, 0.353, 0.000]\n",
      "Epoch 3 [306/340] - Loss: 29.809 [-29.452, 0.357, 0.000]\n",
      "Epoch 3 [307/340] - Loss: 31.212 [-30.853, 0.360, 0.000]\n",
      "Epoch 3 [308/340] - Loss: 27.909 [-27.547, 0.362, 0.000]\n",
      "Epoch 3 [309/340] - Loss: 26.946 [-26.575, 0.371, 0.000]\n",
      "Epoch 3 [310/340] - Loss: 29.342 [-28.962, 0.380, 0.000]\n",
      "Epoch 3 [311/340] - Loss: 30.891 [-30.502, 0.389, 0.000]\n",
      "Epoch 3 [312/340] - Loss: 26.428 [-26.049, 0.379, 0.000]\n",
      "Epoch 3 [313/340] - Loss: 26.013 [-25.630, 0.383, 0.000]\n",
      "Epoch 3 [314/340] - Loss: 28.170 [-27.782, 0.388, 0.000]\n",
      "Epoch 3 [315/340] - Loss: 26.666 [-26.288, 0.378, 0.000]\n",
      "Epoch 3 [316/340] - Loss: 25.568 [-25.164, 0.403, 0.000]\n",
      "Epoch 3 [317/340] - Loss: 29.329 [-28.923, 0.406, 0.000]\n",
      "Epoch 3 [318/340] - Loss: 29.074 [-28.663, 0.412, 0.000]\n",
      "Epoch 3 [319/340] - Loss: 28.519 [-28.108, 0.411, 0.000]\n",
      "Epoch 3 [320/340] - Loss: 26.489 [-26.074, 0.416, 0.000]\n",
      "Epoch 3 [321/340] - Loss: 32.777 [-32.337, 0.440, 0.000]\n",
      "Epoch 3 [322/340] - Loss: 25.534 [-25.066, 0.468, 0.000]\n",
      "Epoch 3 [323/340] - Loss: 31.544 [-31.047, 0.498, 0.000]\n",
      "Epoch 3 [324/340] - Loss: 27.047 [-26.557, 0.490, 0.000]\n",
      "Epoch 3 [325/340] - Loss: 32.362 [-31.890, 0.472, 0.000]\n",
      "Epoch 3 [326/340] - Loss: 28.401 [-27.953, 0.448, 0.000]\n",
      "Epoch 3 [327/340] - Loss: 29.060 [-28.628, 0.432, 0.000]\n",
      "Epoch 3 [328/340] - Loss: 26.708 [-26.288, 0.420, 0.000]\n",
      "Epoch 3 [329/340] - Loss: 30.108 [-29.691, 0.417, 0.000]\n",
      "Epoch 3 [330/340] - Loss: 27.312 [-26.875, 0.437, 0.000]\n",
      "Epoch 3 [331/340] - Loss: 30.774 [-30.329, 0.444, 0.000]\n",
      "Epoch 3 [332/340] - Loss: 25.292 [-24.856, 0.436, 0.000]\n",
      "Epoch 3 [333/340] - Loss: 27.228 [-26.801, 0.427, 0.000]\n",
      "Epoch 3 [334/340] - Loss: 30.417 [-29.997, 0.420, 0.000]\n",
      "Epoch 3 [335/340] - Loss: 27.768 [-27.357, 0.411, 0.000]\n",
      "Epoch 3 [336/340] - Loss: 28.709 [-28.297, 0.412, 0.000]\n",
      "Epoch 3 [337/340] - Loss: 28.167 [-27.752, 0.414, 0.000]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 3 [338/340] - Loss: 28.366 [-27.947, 0.418, 0.000]\n",
      "Epoch 3 [339/340] - Loss: 26.734 [-26.308, 0.427, 0.000]\n",
      "Epoch 4 [0/340] - Loss: 29.253 [-28.815, 0.438, 0.000]\n",
      "Epoch 4 [1/340] - Loss: 26.974 [-26.537, 0.438, 0.000]\n",
      "Epoch 4 [2/340] - Loss: 25.756 [-25.319, 0.437, 0.000]\n",
      "Epoch 4 [3/340] - Loss: 28.685 [-28.250, 0.435, 0.000]\n",
      "Epoch 4 [4/340] - Loss: 26.215 [-25.782, 0.433, 0.000]\n",
      "Epoch 4 [5/340] - Loss: 27.363 [-26.930, 0.434, 0.000]\n",
      "Epoch 4 [6/340] - Loss: 28.645 [-28.217, 0.428, 0.000]\n",
      "Epoch 4 [7/340] - Loss: 29.161 [-28.736, 0.425, 0.000]\n",
      "Epoch 4 [8/340] - Loss: 27.929 [-27.506, 0.423, 0.000]\n",
      "Epoch 4 [9/340] - Loss: 26.959 [-26.539, 0.420, 0.000]\n",
      "Epoch 4 [10/340] - Loss: 25.711 [-25.294, 0.417, 0.000]\n",
      "Epoch 4 [11/340] - Loss: 25.198 [-24.778, 0.420, 0.000]\n",
      "Epoch 4 [12/340] - Loss: 28.243 [-27.830, 0.414, 0.000]\n",
      "Epoch 4 [13/340] - Loss: 27.974 [-27.562, 0.412, 0.000]\n",
      "Epoch 4 [14/340] - Loss: 27.927 [-27.514, 0.412, 0.000]\n",
      "Epoch 4 [15/340] - Loss: 27.779 [-27.367, 0.412, 0.000]\n",
      "Epoch 4 [16/340] - Loss: 26.078 [-25.666, 0.411, 0.000]\n",
      "Epoch 4 [17/340] - Loss: 26.659 [-26.248, 0.411, 0.000]\n",
      "Epoch 4 [18/340] - Loss: 27.645 [-27.232, 0.412, 0.000]\n",
      "Epoch 4 [19/340] - Loss: 27.209 [-26.798, 0.411, 0.000]\n",
      "Epoch 4 [20/340] - Loss: 27.599 [-27.188, 0.411, 0.000]\n",
      "Epoch 4 [21/340] - Loss: 26.537 [-26.126, 0.411, 0.000]\n",
      "Epoch 4 [22/340] - Loss: 27.698 [-27.286, 0.411, 0.000]\n",
      "Epoch 4 [23/340] - Loss: 28.949 [-28.534, 0.414, 0.000]\n",
      "Epoch 4 [24/340] - Loss: 30.299 [-29.880, 0.419, 0.000]\n",
      "Epoch 4 [25/340] - Loss: 26.813 [-26.400, 0.413, 0.000]\n",
      "Epoch 4 [26/340] - Loss: 26.576 [-26.162, 0.414, 0.000]\n",
      "Epoch 4 [27/340] - Loss: 29.229 [-28.813, 0.416, 0.000]\n",
      "Epoch 4 [28/340] - Loss: 27.755 [-27.338, 0.417, 0.000]\n",
      "Epoch 4 [29/340] - Loss: 26.704 [-26.287, 0.417, 0.000]\n",
      "Epoch 4 [30/340] - Loss: 27.497 [-27.077, 0.420, 0.000]\n",
      "Epoch 4 [31/340] - Loss: 25.774 [-25.353, 0.421, 0.000]\n",
      "Epoch 4 [32/340] - Loss: 30.679 [-30.257, 0.423, 0.000]\n",
      "Epoch 4 [33/340] - Loss: 27.809 [-27.383, 0.426, 0.000]\n",
      "Epoch 4 [34/340] - Loss: 25.714 [-25.284, 0.430, 0.000]\n",
      "Epoch 4 [35/340] - Loss: 29.013 [-28.571, 0.442, 0.000]\n",
      "Epoch 4 [36/340] - Loss: 25.169 [-24.733, 0.435, 0.000]\n",
      "Epoch 4 [37/340] - Loss: 25.149 [-24.712, 0.438, 0.000]\n",
      "Epoch 4 [38/340] - Loss: 25.504 [-25.063, 0.441, 0.000]\n",
      "Epoch 4 [39/340] - Loss: 25.530 [-25.086, 0.444, 0.000]\n",
      "Epoch 4 [40/340] - Loss: 27.881 [-27.425, 0.456, 0.000]\n",
      "Epoch 4 [41/340] - Loss: 29.477 [-29.029, 0.448, 0.000]\n",
      "Epoch 4 [42/340] - Loss: 24.360 [-23.900, 0.460, 0.000]\n",
      "Epoch 4 [43/340] - Loss: 27.875 [-27.423, 0.452, 0.000]\n",
      "Epoch 4 [44/340] - Loss: 27.645 [-27.193, 0.452, 0.000]\n",
      "Epoch 4 [45/340] - Loss: 24.676 [-24.223, 0.453, 0.000]\n",
      "Epoch 4 [46/340] - Loss: 28.173 [-27.721, 0.452, 0.000]\n",
      "Epoch 4 [47/340] - Loss: 27.072 [-26.620, 0.452, 0.000]\n",
      "Epoch 4 [48/340] - Loss: 29.414 [-28.963, 0.451, 0.000]\n",
      "Epoch 4 [49/340] - Loss: 27.491 [-27.041, 0.450, 0.000]\n",
      "Epoch 4 [50/340] - Loss: 24.810 [-24.363, 0.448, 0.000]\n",
      "Epoch 4 [51/340] - Loss: 28.700 [-28.252, 0.448, 0.000]\n",
      "Epoch 4 [52/340] - Loss: 28.070 [-27.627, 0.443, 0.000]\n",
      "Epoch 4 [53/340] - Loss: 27.890 [-27.443, 0.447, 0.000]\n",
      "Epoch 4 [54/340] - Loss: 26.259 [-25.819, 0.440, 0.000]\n",
      "Epoch 4 [55/340] - Loss: 27.340 [-26.901, 0.440, 0.000]\n",
      "Epoch 4 [56/340] - Loss: 26.323 [-25.888, 0.436, 0.000]\n",
      "Epoch 4 [57/340] - Loss: 26.814 [-26.380, 0.434, 0.000]\n",
      "Epoch 4 [58/340] - Loss: 27.179 [-26.746, 0.433, 0.000]\n",
      "Epoch 4 [59/340] - Loss: 28.084 [-27.652, 0.433, 0.000]\n",
      "Epoch 4 [60/340] - Loss: 27.570 [-27.132, 0.438, 0.000]\n",
      "Epoch 4 [61/340] - Loss: 27.497 [-27.065, 0.433, 0.000]\n",
      "Epoch 4 [62/340] - Loss: 26.642 [-26.208, 0.434, 0.000]\n",
      "Epoch 4 [63/340] - Loss: 26.902 [-26.471, 0.431, 0.000]\n",
      "Epoch 4 [64/340] - Loss: 29.579 [-29.149, 0.430, 0.000]\n",
      "Epoch 4 [65/340] - Loss: 28.712 [-28.280, 0.431, 0.000]\n",
      "Epoch 4 [66/340] - Loss: 26.223 [-25.794, 0.429, 0.000]\n",
      "Epoch 4 [67/340] - Loss: 27.524 [-27.091, 0.433, 0.000]\n",
      "Epoch 4 [68/340] - Loss: 26.329 [-25.902, 0.428, 0.000]\n",
      "Epoch 4 [69/340] - Loss: 24.472 [-24.043, 0.429, 0.000]\n",
      "Epoch 4 [70/340] - Loss: 26.469 [-26.035, 0.434, 0.000]\n",
      "Epoch 4 [71/340] - Loss: 29.303 [-28.874, 0.430, 0.000]\n",
      "Epoch 4 [72/340] - Loss: 28.296 [-27.868, 0.428, 0.000]\n",
      "Epoch 4 [73/340] - Loss: 27.513 [-27.084, 0.429, 0.000]\n",
      "Epoch 4 [74/340] - Loss: 26.334 [-25.905, 0.429, 0.000]\n",
      "Epoch 4 [75/340] - Loss: 26.898 [-26.469, 0.429, 0.000]\n",
      "Epoch 4 [76/340] - Loss: 28.142 [-27.713, 0.430, 0.000]\n",
      "Epoch 4 [77/340] - Loss: 27.477 [-27.047, 0.430, 0.000]\n",
      "Epoch 4 [78/340] - Loss: 27.208 [-26.778, 0.430, 0.000]\n",
      "Epoch 4 [79/340] - Loss: 27.230 [-26.797, 0.433, 0.000]\n",
      "Epoch 4 [80/340] - Loss: 26.334 [-25.904, 0.430, 0.000]\n",
      "Epoch 4 [81/340] - Loss: 26.604 [-26.163, 0.441, 0.000]\n",
      "Epoch 4 [82/340] - Loss: 28.177 [-27.744, 0.433, 0.000]\n",
      "Epoch 4 [83/340] - Loss: 27.217 [-26.786, 0.431, 0.000]\n",
      "Epoch 4 [84/340] - Loss: 25.668 [-25.237, 0.431, 0.000]\n",
      "Epoch 4 [85/340] - Loss: 27.112 [-26.681, 0.430, 0.000]\n",
      "Epoch 4 [86/340] - Loss: 29.864 [-29.433, 0.431, 0.000]\n",
      "Epoch 4 [87/340] - Loss: 24.524 [-24.093, 0.431, 0.000]\n",
      "Epoch 4 [88/340] - Loss: 27.457 [-27.024, 0.433, 0.000]\n",
      "Epoch 4 [89/340] - Loss: 23.957 [-23.522, 0.435, 0.000]\n",
      "Epoch 4 [90/340] - Loss: 27.410 [-26.980, 0.430, 0.000]\n",
      "Epoch 4 [91/340] - Loss: 24.632 [-24.202, 0.430, 0.000]\n",
      "Epoch 4 [92/340] - Loss: 28.399 [-27.968, 0.431, 0.000]\n",
      "Epoch 4 [93/340] - Loss: 27.910 [-27.479, 0.431, 0.000]\n",
      "Epoch 4 [94/340] - Loss: 23.009 [-22.579, 0.431, 0.000]\n",
      "Epoch 4 [95/340] - Loss: 28.953 [-28.523, 0.430, 0.000]\n",
      "Epoch 4 [96/340] - Loss: 29.018 [-28.588, 0.430, 0.000]\n",
      "Epoch 4 [97/340] - Loss: 27.926 [-27.491, 0.435, 0.000]\n",
      "Epoch 4 [98/340] - Loss: 28.782 [-28.348, 0.434, 0.000]\n",
      "Epoch 4 [99/340] - Loss: 25.488 [-25.058, 0.430, 0.000]\n",
      "Epoch 4 [100/340] - Loss: 27.908 [-27.477, 0.432, 0.000]\n",
      "Epoch 4 [101/340] - Loss: 29.020 [-28.588, 0.433, 0.000]\n",
      "Epoch 4 [102/340] - Loss: 24.920 [-24.490, 0.430, 0.000]\n",
      "Epoch 4 [103/340] - Loss: 27.936 [-27.508, 0.429, 0.000]\n",
      "Epoch 4 [104/340] - Loss: 26.213 [-25.781, 0.432, 0.000]\n",
      "Epoch 4 [105/340] - Loss: 27.071 [-26.644, 0.427, 0.000]\n",
      "Epoch 4 [106/340] - Loss: 30.741 [-30.317, 0.424, 0.000]\n",
      "Epoch 4 [107/340] - Loss: 31.266 [-30.842, 0.424, 0.000]\n",
      "Epoch 4 [108/340] - Loss: 29.442 [-29.018, 0.424, 0.000]\n",
      "Epoch 4 [109/340] - Loss: 30.054 [-29.632, 0.423, 0.000]\n",
      "Epoch 4 [110/340] - Loss: 30.081 [-29.655, 0.426, 0.000]\n",
      "Epoch 4 [111/340] - Loss: 26.232 [-25.805, 0.426, 0.000]\n",
      "Epoch 4 [112/340] - Loss: 23.829 [-23.403, 0.426, 0.000]\n",
      "Epoch 4 [113/340] - Loss: 28.639 [-28.212, 0.427, 0.000]\n",
      "Epoch 4 [114/340] - Loss: 27.358 [-26.933, 0.426, 0.000]\n",
      "Epoch 4 [115/340] - Loss: 24.688 [-24.262, 0.426, 0.000]\n",
      "Epoch 4 [116/340] - Loss: 27.127 [-26.699, 0.427, 0.000]\n",
      "Epoch 4 [117/340] - Loss: 25.994 [-25.556, 0.437, 0.000]\n",
      "Epoch 4 [118/340] - Loss: 25.118 [-24.688, 0.430, 0.000]\n",
      "Epoch 4 [119/340] - Loss: 29.829 [-29.399, 0.430, 0.000]\n",
      "Epoch 4 [120/340] - Loss: 25.402 [-24.972, 0.430, 0.000]\n",
      "Epoch 4 [121/340] - Loss: 24.978 [-24.545, 0.433, 0.000]\n",
      "Epoch 4 [122/340] - Loss: 27.528 [-27.094, 0.434, 0.000]\n",
      "Epoch 4 [123/340] - Loss: 27.878 [-27.448, 0.430, 0.000]\n",
      "Epoch 4 [124/340] - Loss: 28.729 [-28.299, 0.430, 0.000]\n",
      "Epoch 4 [125/340] - Loss: 26.617 [-26.188, 0.429, 0.000]\n",
      "Epoch 4 [126/340] - Loss: 26.521 [-26.086, 0.436, 0.000]\n",
      "Epoch 4 [127/340] - Loss: 26.394 [-25.967, 0.427, 0.000]\n",
      "Epoch 4 [128/340] - Loss: 26.802 [-26.372, 0.430, 0.000]\n",
      "Epoch 4 [129/340] - Loss: 26.310 [-25.882, 0.428, 0.000]\n",
      "Epoch 4 [130/340] - Loss: 29.818 [-29.389, 0.429, 0.000]\n",
      "Epoch 4 [131/340] - Loss: 30.492 [-30.063, 0.429, 0.000]\n",
      "Epoch 4 [132/340] - Loss: 26.954 [-26.525, 0.429, 0.000]\n",
      "Epoch 4 [133/340] - Loss: 27.059 [-26.630, 0.429, 0.000]\n",
      "Epoch 4 [134/340] - Loss: 29.144 [-28.715, 0.428, 0.000]\n",
      "Epoch 4 [135/340] - Loss: 27.384 [-26.956, 0.428, 0.000]\n",
      "Epoch 4 [136/340] - Loss: 27.466 [-27.036, 0.430, 0.000]\n",
      "Epoch 4 [137/340] - Loss: 26.856 [-26.426, 0.430, 0.000]\n",
      "Epoch 4 [138/340] - Loss: 24.891 [-24.462, 0.430, 0.000]\n",
      "Epoch 4 [139/340] - Loss: 26.084 [-25.654, 0.430, 0.000]\n",
      "Epoch 4 [140/340] - Loss: 25.750 [-25.319, 0.431, 0.000]\n",
      "Epoch 4 [141/340] - Loss: 28.333 [-27.904, 0.429, 0.000]\n",
      "Epoch 4 [142/340] - Loss: 24.084 [-23.656, 0.428, 0.000]\n",
      "Epoch 4 [143/340] - Loss: 27.081 [-26.655, 0.426, 0.000]\n",
      "Epoch 4 [144/340] - Loss: 25.300 [-24.869, 0.431, 0.000]\n",
      "Epoch 4 [145/340] - Loss: 27.410 [-26.988, 0.422, 0.000]\n",
      "Epoch 4 [146/340] - Loss: 28.615 [-28.193, 0.422, 0.000]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 4 [147/340] - Loss: 26.593 [-26.172, 0.421, 0.000]\n",
      "Epoch 4 [148/340] - Loss: 27.990 [-27.570, 0.420, 0.000]\n",
      "Epoch 4 [149/340] - Loss: 27.533 [-27.113, 0.420, 0.000]\n",
      "Epoch 4 [150/340] - Loss: 25.909 [-25.489, 0.420, 0.000]\n",
      "Epoch 4 [151/340] - Loss: 26.083 [-25.661, 0.421, 0.000]\n",
      "Epoch 4 [152/340] - Loss: 26.357 [-25.936, 0.422, 0.000]\n",
      "Epoch 4 [153/340] - Loss: 27.813 [-27.383, 0.430, 0.000]\n",
      "Epoch 4 [154/340] - Loss: 25.301 [-24.876, 0.425, 0.000]\n",
      "Epoch 4 [155/340] - Loss: 24.469 [-24.041, 0.428, 0.000]\n",
      "Epoch 4 [156/340] - Loss: 26.029 [-25.602, 0.427, 0.000]\n",
      "Epoch 4 [157/340] - Loss: 28.160 [-27.731, 0.430, 0.000]\n",
      "Epoch 4 [158/340] - Loss: 29.145 [-28.714, 0.431, 0.000]\n",
      "Epoch 4 [159/340] - Loss: 25.986 [-25.553, 0.432, 0.000]\n",
      "Epoch 4 [160/340] - Loss: 26.463 [-26.029, 0.434, 0.000]\n",
      "Epoch 4 [161/340] - Loss: 27.314 [-26.879, 0.436, 0.000]\n",
      "Epoch 4 [162/340] - Loss: 22.410 [-21.974, 0.436, 0.000]\n",
      "Epoch 4 [163/340] - Loss: 23.168 [-22.731, 0.437, 0.000]\n",
      "Epoch 4 [164/340] - Loss: 27.549 [-27.108, 0.441, 0.000]\n",
      "Epoch 4 [165/340] - Loss: 28.279 [-27.839, 0.439, 0.000]\n",
      "Epoch 4 [166/340] - Loss: 28.213 [-27.774, 0.440, 0.000]\n",
      "Epoch 4 [167/340] - Loss: 26.780 [-26.342, 0.439, 0.000]\n",
      "Epoch 4 [168/340] - Loss: 26.172 [-25.733, 0.439, 0.000]\n",
      "Epoch 4 [169/340] - Loss: 25.976 [-25.537, 0.439, 0.000]\n",
      "Epoch 4 [170/340] - Loss: 27.946 [-27.501, 0.444, 0.000]\n",
      "Epoch 4 [171/340] - Loss: 25.507 [-25.067, 0.439, 0.000]\n",
      "Epoch 4 [172/340] - Loss: 27.787 [-27.342, 0.445, 0.000]\n",
      "Epoch 4 [173/340] - Loss: 26.954 [-26.511, 0.443, 0.000]\n",
      "Epoch 4 [174/340] - Loss: 29.088 [-28.649, 0.438, 0.000]\n",
      "Epoch 4 [175/340] - Loss: 25.975 [-25.534, 0.441, 0.000]\n",
      "Epoch 4 [176/340] - Loss: 29.341 [-28.903, 0.438, 0.000]\n",
      "Epoch 4 [177/340] - Loss: 25.318 [-24.881, 0.436, 0.000]\n",
      "Epoch 4 [178/340] - Loss: 27.275 [-26.840, 0.435, 0.000]\n",
      "Epoch 4 [179/340] - Loss: 26.322 [-25.890, 0.433, 0.000]\n",
      "Epoch 4 [180/340] - Loss: 26.742 [-26.310, 0.432, 0.000]\n",
      "Epoch 4 [181/340] - Loss: 24.988 [-24.557, 0.430, 0.000]\n",
      "Epoch 4 [182/340] - Loss: 27.652 [-27.223, 0.429, 0.000]\n",
      "Epoch 4 [183/340] - Loss: 26.610 [-26.182, 0.428, 0.000]\n",
      "Epoch 4 [184/340] - Loss: 25.384 [-24.957, 0.427, 0.000]\n",
      "Epoch 4 [185/340] - Loss: 25.091 [-24.665, 0.426, 0.000]\n",
      "Epoch 4 [186/340] - Loss: 27.524 [-27.097, 0.427, 0.000]\n",
      "Epoch 4 [187/340] - Loss: 25.306 [-24.881, 0.425, 0.000]\n",
      "Epoch 4 [188/340] - Loss: 25.457 [-25.032, 0.424, 0.000]\n",
      "Epoch 4 [189/340] - Loss: 27.343 [-26.921, 0.421, 0.000]\n",
      "Epoch 4 [190/340] - Loss: 27.687 [-27.266, 0.421, 0.000]\n",
      "Epoch 4 [191/340] - Loss: 25.122 [-24.705, 0.417, 0.000]\n",
      "Epoch 4 [192/340] - Loss: 29.868 [-29.452, 0.416, 0.000]\n",
      "Epoch 4 [193/340] - Loss: 25.488 [-25.073, 0.415, 0.000]\n",
      "Epoch 4 [194/340] - Loss: 25.045 [-24.630, 0.415, 0.000]\n",
      "Epoch 4 [195/340] - Loss: 24.081 [-23.665, 0.415, 0.000]\n",
      "Epoch 4 [196/340] - Loss: 28.504 [-28.084, 0.420, 0.000]\n",
      "Epoch 4 [197/340] - Loss: 24.757 [-24.342, 0.416, 0.000]\n",
      "Epoch 4 [198/340] - Loss: 27.055 [-26.639, 0.416, 0.000]\n",
      "Epoch 4 [199/340] - Loss: 27.205 [-26.787, 0.418, 0.000]\n",
      "Epoch 4 [200/340] - Loss: 27.187 [-26.763, 0.424, 0.000]\n",
      "Epoch 4 [201/340] - Loss: 24.749 [-24.328, 0.421, 0.000]\n",
      "Epoch 4 [202/340] - Loss: 26.258 [-25.838, 0.421, 0.000]\n",
      "Epoch 4 [203/340] - Loss: 30.399 [-29.976, 0.423, 0.000]\n",
      "Epoch 4 [204/340] - Loss: 24.670 [-24.245, 0.425, 0.000]\n",
      "Epoch 4 [205/340] - Loss: 26.688 [-26.261, 0.427, 0.000]\n",
      "Epoch 4 [206/340] - Loss: 29.512 [-29.082, 0.430, 0.000]\n",
      "Epoch 4 [207/340] - Loss: 23.794 [-23.363, 0.431, 0.000]\n",
      "Epoch 4 [208/340] - Loss: 25.568 [-25.133, 0.435, 0.000]\n",
      "Epoch 4 [209/340] - Loss: 25.381 [-24.945, 0.436, 0.000]\n",
      "Epoch 4 [210/340] - Loss: 25.474 [-25.031, 0.443, 0.000]\n",
      "Epoch 4 [211/340] - Loss: 26.297 [-25.856, 0.441, 0.000]\n",
      "Epoch 4 [212/340] - Loss: 28.925 [-28.484, 0.441, 0.000]\n",
      "Epoch 4 [213/340] - Loss: 27.944 [-27.496, 0.447, 0.000]\n",
      "Epoch 4 [214/340] - Loss: 26.547 [-26.106, 0.441, 0.000]\n",
      "Epoch 4 [215/340] - Loss: 25.218 [-24.780, 0.438, 0.000]\n",
      "Epoch 4 [216/340] - Loss: 25.255 [-24.820, 0.435, 0.000]\n",
      "Epoch 4 [217/340] - Loss: 29.481 [-29.049, 0.432, 0.000]\n",
      "Epoch 4 [218/340] - Loss: 27.443 [-27.013, 0.431, 0.000]\n",
      "Epoch 4 [219/340] - Loss: 25.019 [-24.592, 0.427, 0.000]\n",
      "Epoch 4 [220/340] - Loss: 24.922 [-24.497, 0.424, 0.000]\n",
      "Epoch 4 [221/340] - Loss: 27.998 [-27.574, 0.424, 0.000]\n",
      "Epoch 4 [222/340] - Loss: 24.749 [-24.327, 0.422, 0.000]\n",
      "Epoch 4 [223/340] - Loss: 26.939 [-26.518, 0.421, 0.000]\n",
      "Epoch 4 [224/340] - Loss: 26.374 [-25.955, 0.420, 0.000]\n",
      "Epoch 4 [225/340] - Loss: 23.937 [-23.517, 0.419, 0.000]\n",
      "Epoch 4 [226/340] - Loss: 29.403 [-28.983, 0.420, 0.000]\n",
      "Epoch 4 [227/340] - Loss: 26.897 [-26.477, 0.420, 0.000]\n",
      "Epoch 4 [228/340] - Loss: 27.327 [-26.906, 0.422, 0.000]\n",
      "Epoch 4 [229/340] - Loss: 28.006 [-27.582, 0.424, 0.000]\n",
      "Epoch 4 [230/340] - Loss: 27.018 [-26.594, 0.424, 0.000]\n",
      "Epoch 4 [231/340] - Loss: 26.810 [-26.383, 0.427, 0.000]\n",
      "Epoch 4 [232/340] - Loss: 26.274 [-25.844, 0.430, 0.000]\n",
      "Epoch 4 [233/340] - Loss: 28.482 [-28.047, 0.435, 0.000]\n",
      "Epoch 4 [234/340] - Loss: 26.243 [-25.803, 0.439, 0.000]\n",
      "Epoch 4 [235/340] - Loss: 25.798 [-25.359, 0.439, 0.000]\n",
      "Epoch 4 [236/340] - Loss: 26.081 [-25.634, 0.446, 0.000]\n",
      "Epoch 4 [237/340] - Loss: 24.693 [-24.247, 0.446, 0.000]\n",
      "Epoch 4 [238/340] - Loss: 25.818 [-25.368, 0.450, 0.000]\n",
      "Epoch 4 [239/340] - Loss: 25.361 [-24.913, 0.448, 0.000]\n",
      "Epoch 4 [240/340] - Loss: 23.911 [-23.457, 0.454, 0.000]\n",
      "Epoch 4 [241/340] - Loss: 27.842 [-27.392, 0.451, 0.000]\n",
      "Epoch 4 [242/340] - Loss: 27.409 [-26.958, 0.451, 0.000]\n",
      "Epoch 4 [243/340] - Loss: 27.395 [-26.946, 0.450, 0.000]\n",
      "Epoch 4 [244/340] - Loss: 24.462 [-24.014, 0.449, 0.000]\n",
      "Epoch 4 [245/340] - Loss: 28.144 [-27.698, 0.447, 0.000]\n",
      "Epoch 4 [246/340] - Loss: 25.069 [-24.627, 0.442, 0.000]\n",
      "Epoch 4 [247/340] - Loss: 26.043 [-25.603, 0.440, 0.000]\n",
      "Epoch 4 [248/340] - Loss: 25.274 [-24.833, 0.441, 0.000]\n",
      "Epoch 4 [249/340] - Loss: 27.109 [-26.673, 0.437, 0.000]\n",
      "Epoch 4 [250/340] - Loss: 25.547 [-25.111, 0.436, 0.000]\n",
      "Epoch 4 [251/340] - Loss: 28.462 [-28.029, 0.433, 0.000]\n",
      "Epoch 4 [252/340] - Loss: 26.616 [-26.185, 0.431, 0.000]\n",
      "Epoch 4 [253/340] - Loss: 25.383 [-24.953, 0.430, 0.000]\n",
      "Epoch 4 [254/340] - Loss: 24.576 [-24.147, 0.429, 0.000]\n",
      "Epoch 4 [255/340] - Loss: 26.582 [-26.153, 0.429, 0.000]\n",
      "Epoch 4 [256/340] - Loss: 30.546 [-30.117, 0.429, 0.000]\n",
      "Epoch 4 [257/340] - Loss: 29.276 [-28.846, 0.430, 0.000]\n",
      "Epoch 4 [258/340] - Loss: 25.308 [-24.877, 0.431, 0.000]\n",
      "Epoch 4 [259/340] - Loss: 27.653 [-27.222, 0.431, 0.000]\n",
      "Epoch 4 [260/340] - Loss: 27.380 [-26.942, 0.438, 0.000]\n",
      "Epoch 4 [261/340] - Loss: 27.051 [-26.618, 0.433, 0.000]\n",
      "Epoch 4 [262/340] - Loss: 26.984 [-26.550, 0.434, 0.000]\n",
      "Epoch 4 [263/340] - Loss: 28.174 [-27.739, 0.434, 0.000]\n",
      "Epoch 4 [264/340] - Loss: 25.470 [-25.032, 0.439, 0.000]\n",
      "Epoch 4 [265/340] - Loss: 23.428 [-22.990, 0.438, 0.000]\n",
      "Epoch 4 [266/340] - Loss: 22.813 [-22.374, 0.439, 0.000]\n",
      "Epoch 4 [267/340] - Loss: 25.084 [-24.645, 0.438, 0.000]\n",
      "Epoch 4 [268/340] - Loss: 24.670 [-24.232, 0.438, 0.000]\n",
      "Epoch 4 [269/340] - Loss: 25.148 [-24.711, 0.437, 0.000]\n",
      "Epoch 4 [270/340] - Loss: 23.924 [-23.487, 0.436, 0.000]\n",
      "Epoch 4 [271/340] - Loss: 26.098 [-25.663, 0.436, 0.000]\n",
      "Epoch 4 [272/340] - Loss: 24.351 [-23.916, 0.435, 0.000]\n",
      "Epoch 4 [273/340] - Loss: 26.595 [-26.162, 0.433, 0.000]\n",
      "Epoch 4 [274/340] - Loss: 25.291 [-24.854, 0.437, 0.000]\n",
      "Epoch 4 [275/340] - Loss: 26.089 [-25.655, 0.434, 0.000]\n",
      "Epoch 4 [276/340] - Loss: 27.861 [-27.430, 0.431, 0.000]\n",
      "Epoch 4 [277/340] - Loss: 25.012 [-24.580, 0.431, 0.000]\n",
      "Epoch 4 [278/340] - Loss: 25.478 [-25.046, 0.433, 0.000]\n",
      "Epoch 4 [279/340] - Loss: 26.477 [-26.038, 0.439, 0.000]\n",
      "Epoch 4 [280/340] - Loss: 26.609 [-26.170, 0.439, 0.000]\n",
      "Epoch 4 [281/340] - Loss: 27.163 [-26.727, 0.437, 0.000]\n",
      "Epoch 4 [282/340] - Loss: 25.875 [-25.436, 0.438, 0.000]\n",
      "Epoch 4 [283/340] - Loss: 26.301 [-25.858, 0.443, 0.000]\n",
      "Epoch 4 [284/340] - Loss: 24.411 [-23.968, 0.442, 0.000]\n",
      "Epoch 4 [285/340] - Loss: 28.514 [-28.066, 0.448, 0.000]\n",
      "Epoch 4 [286/340] - Loss: 25.664 [-25.218, 0.446, 0.000]\n",
      "Epoch 4 [287/340] - Loss: 26.831 [-26.384, 0.447, 0.000]\n",
      "Epoch 4 [288/340] - Loss: 25.343 [-24.897, 0.446, 0.000]\n",
      "Epoch 4 [289/340] - Loss: 26.746 [-26.300, 0.446, 0.000]\n",
      "Epoch 4 [290/340] - Loss: 24.167 [-23.723, 0.445, 0.000]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 4 [291/340] - Loss: 26.725 [-26.281, 0.444, 0.000]\n",
      "Epoch 4 [292/340] - Loss: 26.811 [-26.369, 0.442, 0.000]\n",
      "Epoch 4 [293/340] - Loss: 24.623 [-24.181, 0.442, 0.000]\n",
      "Epoch 4 [294/340] - Loss: 27.364 [-26.918, 0.446, 0.000]\n",
      "Epoch 4 [295/340] - Loss: 27.307 [-26.864, 0.443, 0.000]\n",
      "Epoch 4 [296/340] - Loss: 24.420 [-23.978, 0.442, 0.000]\n",
      "Epoch 4 [297/340] - Loss: 28.476 [-28.033, 0.442, 0.000]\n",
      "Epoch 4 [298/340] - Loss: 25.343 [-24.900, 0.443, 0.000]\n",
      "Epoch 4 [299/340] - Loss: 23.952 [-23.508, 0.444, 0.000]\n",
      "Epoch 4 [300/340] - Loss: 27.814 [-27.368, 0.446, 0.000]\n",
      "Epoch 4 [301/340] - Loss: 25.337 [-24.892, 0.444, 0.000]\n",
      "Epoch 4 [302/340] - Loss: 27.782 [-27.336, 0.446, 0.000]\n",
      "Epoch 4 [303/340] - Loss: 27.219 [-26.775, 0.444, 0.000]\n",
      "Epoch 4 [304/340] - Loss: 25.723 [-25.280, 0.443, 0.000]\n",
      "Epoch 4 [305/340] - Loss: 29.251 [-28.809, 0.442, 0.000]\n",
      "Epoch 4 [306/340] - Loss: 26.144 [-25.702, 0.441, 0.000]\n",
      "Epoch 4 [307/340] - Loss: 29.418 [-28.976, 0.442, 0.000]\n",
      "Epoch 4 [308/340] - Loss: 26.509 [-26.067, 0.442, 0.000]\n",
      "Epoch 4 [309/340] - Loss: 25.468 [-25.021, 0.446, 0.000]\n",
      "Epoch 4 [310/340] - Loss: 25.060 [-24.612, 0.448, 0.000]\n",
      "Epoch 4 [311/340] - Loss: 26.715 [-26.272, 0.442, 0.000]\n",
      "Epoch 4 [312/340] - Loss: 25.896 [-25.453, 0.443, 0.000]\n",
      "Epoch 4 [313/340] - Loss: 25.882 [-25.440, 0.442, 0.000]\n",
      "Epoch 4 [314/340] - Loss: 25.263 [-24.822, 0.440, 0.000]\n",
      "Epoch 4 [315/340] - Loss: 24.532 [-24.093, 0.439, 0.000]\n",
      "Epoch 4 [316/340] - Loss: 26.268 [-25.830, 0.439, 0.000]\n",
      "Epoch 4 [317/340] - Loss: 27.893 [-27.456, 0.437, 0.000]\n",
      "Epoch 4 [318/340] - Loss: 28.316 [-27.880, 0.436, 0.000]\n",
      "Epoch 4 [319/340] - Loss: 23.482 [-23.049, 0.434, 0.000]\n",
      "Epoch 4 [320/340] - Loss: 23.633 [-23.202, 0.431, 0.000]\n",
      "Epoch 4 [321/340] - Loss: 26.331 [-25.902, 0.429, 0.000]\n",
      "Epoch 4 [322/340] - Loss: 25.024 [-24.596, 0.428, 0.000]\n",
      "Epoch 4 [323/340] - Loss: 25.393 [-24.967, 0.427, 0.000]\n",
      "Epoch 4 [324/340] - Loss: 23.961 [-23.529, 0.432, 0.000]\n",
      "Epoch 4 [325/340] - Loss: 29.166 [-28.741, 0.425, 0.000]\n",
      "Epoch 4 [326/340] - Loss: 25.725 [-25.302, 0.424, 0.000]\n",
      "Epoch 4 [327/340] - Loss: 23.426 [-23.002, 0.424, 0.000]\n",
      "Epoch 4 [328/340] - Loss: 24.911 [-24.486, 0.425, 0.000]\n",
      "Epoch 4 [329/340] - Loss: 28.765 [-28.340, 0.425, 0.000]\n",
      "Epoch 4 [330/340] - Loss: 26.373 [-25.947, 0.426, 0.000]\n",
      "Epoch 4 [331/340] - Loss: 29.779 [-29.352, 0.428, 0.000]\n",
      "Epoch 4 [332/340] - Loss: 25.147 [-24.718, 0.429, 0.000]\n",
      "Epoch 4 [333/340] - Loss: 27.783 [-27.351, 0.431, 0.000]\n",
      "Epoch 4 [334/340] - Loss: 24.223 [-23.789, 0.434, 0.000]\n",
      "Epoch 4 [335/340] - Loss: 27.552 [-27.115, 0.436, 0.000]\n",
      "Epoch 4 [336/340] - Loss: 29.308 [-28.868, 0.441, 0.000]\n",
      "Epoch 4 [337/340] - Loss: 24.854 [-24.410, 0.444, 0.000]\n",
      "Epoch 4 [338/340] - Loss: 26.236 [-25.793, 0.444, 0.000]\n",
      "Epoch 4 [339/340] - Loss: 27.786 [-27.341, 0.445, 0.000]\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "model.train()\n",
    "likelihood.train()\n",
    "\n",
    "# We'll do 6 epochs of training in this tutorial\n",
    "num_epochs = 4\n",
    "\n",
    "# We use SGD here, rather than Adam. Emperically, we find that SGD is better for variational regression\n",
    "optimizer = torch.optim.Adam([\n",
    "    {'params': model.parameters()},\n",
    "    {'params': likelihood.parameters()},\n",
    "], lr=0.01)\n",
    "\n",
    "scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones=[3, 5], gamma=0.1)\n",
    "\n",
    "# Our loss object. We're using the VariationalELBO, which essentially just computes the ELBO\n",
    "mll = gpytorch.mlls.VariationalELBO(likelihood, model, num_data=train_y.size(0), combine_terms=False)\n",
    "\n",
    "# We use more CG iterations here because the preconditioner introduced in the NeurIPS paper seems to be less\n",
    "# effective for VI.\n",
    "with gpytorch.settings.max_cg_iterations(45):\n",
    "    for i in range(num_epochs):\n",
    "        scheduler.step()\n",
    "        # Within each iteration, we will go over each minibatch of data\n",
    "        for minibatch_i, (x_batch, y_batch) in enumerate(train_loader):\n",
    "            start_time = time.time()\n",
    "            optimizer.zero_grad()\n",
    "            output = model(x_batch)\n",
    "            # with combine_terms=False, we get the terms of the ELBO separated so we can print them individually if we'd like.\n",
    "            # loss = -mll(output, y_batch) would also work.\n",
    "            log_lik, kl_div, log_prior = mll(output, y_batch)\n",
    "            loss = -(log_lik - kl_div + log_prior)\n",
    "            print('Epoch %d [%d/%d] - Loss: %.3f [%.3f, %.3f, %.3f]' % (i + 1, minibatch_i, len(train_loader), loss.item(), log_lik.item(), kl_div.item(), log_prior.item()))\n",
    "\n",
    "            loss.backward()\n",
    "            optimizer.step()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making Predictions\n",
    "\n",
    "The next cell gets the predictive covariance for the test set (and also technically gets the predictive mean, stored in `preds.mean()`). Because the test set is substantially smaller than the training set, we don't need to make predictions in mini batches here, although this can be done by passing in minibatches of `test_x` rather than the full tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jake.gardner/git/gpytorch/gpytorch/lazy/cached_cg_lazy_tensor.py:161: UserWarning: CachedCGLazyTensor had to run CG on a tensor of size torch.Size([500, 1918]). For best performance, this LazyTensor should pre-register all vectors to run CG against.\n",
      "  \"LazyTensor should pre-register all vectors to run CG against.\".format(rhs.shape)\n"
     ]
    }
   ],
   "source": [
    "model.eval()\n",
    "likelihood.eval()\n",
    "means = torch.tensor([0.])\n",
    "with torch.no_grad():\n",
    "    for x_batch, y_batch in test_loader:\n",
    "        preds = model(x_batch)\n",
    "        means = torch.cat([means, preds.mean.cpu()])\n",
    "means = means[1:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test MAE: 9.761589050292969\n"
     ]
    }
   ],
   "source": [
    "print('Test MAE: {}'.format(torch.mean(torch.abs(means - test_y.cpu()))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
